x86 : Identifying Function Calling Conventions From Assembly Code
In the previous post x86 : Function Calling Conventions we have seen the tricks which can be used to identify which calling function is being used for a function. In this article we will debug a program which will let us exercise that knowledge in a real debugging session.
Commands used:
0:000> x DiffCallingConventions!*func*
00241030 DiffCallingConventions!Function1 (unsigned int, unsigned int, unsigned int, unsigned int)
00241070 DiffCallingConventions!Function3 (unsigned int, unsigned int, unsigned int, unsigned int)
00241050 DiffCallingConventions!Function2 (unsigned int, unsigned int, unsigned int, unsigned int)
This above output shows that there are three functions with the same parameters. lets see if we can identify their calling conventions by disassembling them.
0:000> uf DiffCallingConventions!Function1
DiffCallingConventions!Function1 :
14 00241030 55 push ebp
14 00241031 8bec mov ebp,esp
14 00241033 51 push ecx
18 00241034 8b4508 mov eax,dword ptr [ebp+8]
18 00241037 03450c add eax,dword ptr [ebp+0Ch]
18 0024103a 034510 add eax,dword ptr [ebp+10h]
18 0024103d 034514 add eax,dword ptr [ebp+14h]
18 00241040 8945fc mov dword ptr [ebp-4],eax
20 00241043 8b45fc mov eax,dword ptr [ebp-4]
22 00241046 8be5 mov esp,ebp
22 00241048 5d pop ebp
22 00241049 c3 ret
We know that only the CDECL calling convention uses the ret instruction without any parameter. So this definitely is the standard calling convention.
0:000> uf DiffCallingConventions!Function2
DiffCallingConventions!Function2 :
34 00241050 55 push ebp
34 00241051 8bec mov ebp,esp
34 00241053 51 push ecx
38 00241054 8b4508 mov eax,dword ptr [ebp+8]
38 00241057 03450c add eax,dword ptr [ebp+0Ch]
38 0024105a 034510 add eax,dword ptr [ebp+10h]
38 0024105d 034514 add eax,dword ptr [ebp+14h]
38 00241060 8945fc mov dword ptr [ebp-4],eax
40 00241063 8b45fc mov eax,dword ptr [ebp-4]
42 00241066 8be5 mov esp,ebp
42 00241068 5d pop ebp
42 00241069 c21000 ret 10h
Okay for the second function ret does take a parameter, so it is not standard call, but it can be either this call or fast call or stdcall.
The line in bold above shows that the register ecx is being used in the function without being initialized. This will only work if the value of ecx is already initialized by the caller. We know that FASTCALL uses registers to pass the first 4 parameters to a function. Hence registers ecx and edx are set to those parameters. Thus we can conclude that this is FASTCALL.
0:000> uf DiffCallingConventions!Function3
DiffCallingConventions!Function3:
53 00241070 55 push ebp
53 00241071 8bec mov ebp,esp
53 00241073 83ec0c sub esp,0Ch
53 00241076 8955f8 mov dword ptr [ebp-8],edx
53 00241079 894dfc mov dword ptr [ebp-4],ecx
57 0024107c 8b45fc mov eax,dword ptr [ebp-4]
57 0024107f 0345f8 add eax,dword ptr [ebp-8]
57 00241082 034508 add eax,dword ptr [ebp+8]
57 00241085 03450c add eax,dword ptr [ebp+0Ch]
57 00241088 8945f4 mov dword ptr [ebp-0Ch],eax
59 0024108b 8b45f4 mov eax,dword ptr [ebp-0Ch]
61 0024108e 8be5 mov esp,ebp
61 00241090 5d pop ebp
61 00241091 c20800 ret 8
Function 3 seems to be using a parameter in ret, but it also doesnt uses ecx or edx without initializing. This indicates that this function is setup using stdcall.
Hope this helps you debug your programs in the future.
No comments:
Post a Comment