WinDbg : Debugging A Critical Section Based Dead Lock Scenario

WinDbg : Debugging A Critical Section Based Dead Lock Scenario

Dead locks can be very nasty and sometimes very hard to debug and root cause. This post is an attempt to walk you through one such scenario, where a critical section dead locks a thread. As with most of the previous posts, we will refrain from having source level help, that is, we will try and debug the faulty application without any source code and only symbols. 

Commands Used:

1. k
2. uF and u
3. dd

We start by attaching the debugger to he hung application, or we can also start the application inside the debugger if that is an alternative. In this example the name of the application is FaultyApp.Exe.

Once the application hangs, we start by taking a stack dump of all the threads currently running. Since this is an user land debugging scenario, the ~* command will give us the thread wise stack dump.

kd>~*kvn

0:004> ~*kvn

   0  Id: 6a0.5f8 Suspend: 1 Teb: 7efdd000 Unfrozen

 # ChildEBP RetAddr  Args to Child              

00 0016f7b8 770315f7 00000003 0016f808 00000001 ntdll!NtWaitForMultipleObjects+0x15 (FPO: [5,0,0])

01 0016f854 76e719f8 0016f808 0016f87c 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x100 (FPO: [Non-Fpo])

02 0016f89c 76e74200 00000003 7efde000 00000000 kernel32!WaitForMultipleObjectsExImplementation+0xe0 (FPO: [Non-Fpo])

03 0016f8b8 00bd11b5 00000003 0016f8d4 00000000 kernel32!WaitForMultipleObjects+0x18 (FPO: [Non-Fpo])

04 0016f8e4 00bd143d 00000001 00699318 00699368 FaultyApp!main+0x95 (FPO: [Non-Fpo]) (CONV: cdecl) 

05 0016f92c 76e7338a 7efde000 0016f978 77d79f72 FaultyApp!__tmainCRTStartup+0xfe (FPO: [Non-Fpo]) (CONV: cdecl) 

06 0016f938 77d79f72 7efde000 735ac1c9 00000000 kernel32!BaseThreadInitThunk+0xe (FPO: [Non-Fpo])

07 0016f978 77d79f45 00bd1500 7efde000 ffffffff ntdll!__RtlUserThreadStart+0x70 (FPO: [Non-Fpo])

08 0016f990 00000000 00bd1500 7efde000 00000000 ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])

   1  Id: 6a0.bc4 Suspend: 1 Teb: 7efda000 Unfrozen

 # ChildEBP RetAddr  Args to Child              

00 0089feb4 77d78e44 00000048 00000000 00000000 ntdll!ZwWaitForSingleObject+0x15 (FPO: [3,0,0])

01 0089ff18 77d78d28 00000000 00000000 00000000 ntdll!RtlpWaitOnCriticalSection+0x13e (FPO: [Non-Fpo])

02 0089ff40 00bd105c 6ce392e0 00000000 00000000 ntdll!RtlEnterCriticalSection+0x150 (FPO: [Non-Fpo])

03 0089ff54 76e7338a 00000000 0089ffa0 77d79f72 FaultyApp!FnThread1+0x2c (FPO: [Non-Fpo]) (CONV: cdecl) 

04 0089ff60 77d79f72 00000000 73c5c711 00000000 kernel32!BaseThreadInitThunk+0xe (FPO: [Non-Fpo])

05 0089ffa0 77d79f45 00bd1005 00000000 ffffffff ntdll!__RtlUserThreadStart+0x70 (FPO: [Non-Fpo])

06 0089ffb8 00000000 00bd1005 00000000 00000000 ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])

   2  Id: 6a0.12fc Suspend: 1 Teb: 7efaf000 Unfrozen

 # ChildEBP RetAddr  Args to Child              

00 00dffa50 77d78e44 00000048 00000000 00000000 ntdll!ZwWaitForSingleObject+0x15 (FPO: [3,0,0])

01 00dffab4 77d78d28 00000000 00000000 00000000 ntdll!RtlpWaitOnCriticalSection+0x13e (FPO: [Non-Fpo])

02 00dffadc 00bd10fc 6ce392e0 00000000 00000000 ntdll!RtlEnterCriticalSection+0x150 (FPO: [Non-Fpo])

03 00dffaf0 76e7338a 00000000 00dffb3c 77d79f72 FaultyApp!FnThread3+0x2c (FPO: [Non-Fpo]) (CONV: cdecl) 

04 00dffafc 77d79f72 00000000 7393c38d 00000000 kernel32!BaseThreadInitThunk+0xe (FPO: [Non-Fpo])

05 00dffb3c 77d79f45 00bd100f 00000000 ffffffff ntdll!__RtlUserThreadStart+0x70 (FPO: [Non-Fpo])

06 00dffb54 00000000 00bd100f 00000000 00000000 ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])

   3  Id: 6a0.11f0 Suspend: 1 Teb: 7efd7000 Unfrozen

 # ChildEBP RetAddr  Args to Child              

00 00a6fa34 77d78e44 00000048 00000000 00000000 ntdll!ZwWaitForSingleObject+0x15 (FPO: [3,0,0])

01 00a6fa98 77d78d28 00000000 00000000 00000000 ntdll!RtlpWaitOnCriticalSection+0x13e (FPO: [Non-Fpo])

02 00a6fac0 00bd10ac 6ce392e0 00000000 00000000 ntdll!RtlEnterCriticalSection+0x150 (FPO: [Non-Fpo])

03 00a6fad4 76e7338a 00000000 00a6fb20 77d79f72 FaultyApp!FnThread2+0x2c (FPO: [Non-Fpo]) (CONV: cdecl) 

04 00a6fae0 77d79f72 00000000 73eac391 00000000 kernel32!BaseThreadInitThunk+0xe (FPO: [Non-Fpo])

05 00a6fb20 77d79f45 00bd100a 00000000 ffffffff ntdll!__RtlUserThreadStart+0x70 (FPO: [Non-Fpo])

06 00a6fb38 00000000 00bd100a 00000000 00000000 ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])

#  4  Id: 6a0.11c8 Suspend: 1 Teb: 7efac000 Unfrozen

 # ChildEBP RetAddr  Args to Child              

00 00f0fd3c 77ddf926 73bcc5dd 00000000 00000000 ntdll!DbgBreakPoint (FPO: [0,0,0])

01 00f0fd6c 76e7338a 00000000 00f0fdb8 77d79f72 ntdll!DbgUiRemoteBreakin+0x3c (FPO: [Non-Fpo])

02 00f0fd78 77d79f72 00000000 73bcc509 00000000 kernel32!BaseThreadInitThunk+0xe (FPO: [Non-Fpo])

03 00f0fdb8 77d79f45 77ddf8ea 00000000 ffffffff ntdll!__RtlUserThreadStart+0x70 (FPO: [Non-Fpo])

04 00f0fdd0 00000000 77ddf8ea 00000000 00000000 ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])

The clue here is to look for the Windows API call RtlEnterCriticalSection. There might be  more than one thread in the stack making this function call, we can use any such thread.

Looking up on MSDN we see that the API is defined as follows:

NTSTATUS RtlEnterCriticalSection

 (

  RTL_CRITICAL_SECTION* crit

 )

In the above stack dump, we have several such threads, we take any one and them from the parameters list take the address of the RTL_CRITICAL_SECTION object pointer and type cast it to see the contents.

02 0089ff40 00bd105c 6ce392e0 00000000 00000000 ntdll!RtlEnterCriticalSection+0x150 (FPO: [Non-Fpo])

0:004> dt ntdll!_RTL_CRITICAL_SECTION 6ce392e0

   +0x000 DebugInfo        : 0x0069acf8 _RTL_CRITICAL_SECTION_DEBUG

   +0x004 LockCount        : 0n-14

   +0x008 RecursionCount   : 0n1

   +0x00c OwningThread     : 0x000005f8 Void

   +0x010 LockSemaphore    : 0x00000048 Void

   +0x014 SpinCount        : 0

We see that one of the members of this object gives us the Owning thread ID. 0x000005f8.

Lets try and find out more details about this thread. In user mode, the double tilde (~~) command will do it for us.

0:004> ~~[0x000005f8 ]

   0  Id: 6a0.5f8 Suspend: 1 Teb: 7efdd000 Unfrozen

      Start: FaultyApp!mainCRTStartup (00bd1500)

      Priority: 0  Priority class: 32  Affinity: ff

Ok so the thread offending is the main thread. This is good. Now we know where in the code of FaultyApp we need to start digging.

Lets start with disassembling the Main function. I will add the code in small sections explaining the relevant portions in between.

0:004> uF FaultyApp!main

FaultyApp!main:

00bd1120 55              push    ebp

00bd1121 8bec           mov    ebp,esp 

This is standard function prologue code, that we see in the above two lines.

00bd1123 83ec14        sub    esp,14h                                           

The above line is allocating space for local variables in the stack 

00bd1126 a11070bf00  mov   eax,dword ptr [FaultyApp!__security_cookie (00bf7010)]

00bd112b 33c5            xor     eax,ebp                                           

These two lines above is the code for stack guard.

00bd112d 8945fc          mov     dword ptr [ebp-4],eax

00bd1130 a100b0bf00   mov     eax,dword ptr [FaultyApp!_imp__g_CritSec (00bfb000)]

00bd1135 50                 push    eax

00bd1136 ff1534b0bf00 call    dword ptr [FaultyApp!_imp__InitializeCriticalSection (00bfb034)]    

the four lines above  is the code to initialize the critical section

00bd113c c745ec00000000  mov     dword ptr [ebp-14h],0

00bd1143 eb09                   jmp     FaultyApp!main+0x2e (00bd114e)

FaultyApp!main+0x25:

00bd1145 8b4dec          mov     ecx,dword ptr [ebp-14h]

00bd1148 83c101          add      ecx,1

00bd114b 894dec          mov     dword ptr [ebp-14h],ecx

FaultyApp!main+0x2e:

00bd114e 837dec03        cmp     dword ptr [ebp-14h],3                             

The line just above is comparing a local variable with a constant '3'

00bd1152 7333            jae     FaultyApp!main+0x67 (00bd1187)                  

If the comparison result is equal or greater than 3 then then jump out, else loop                                                    

FaultyApp!main+0x34:

00bd1154 6a00                    push    0

00bd1156 6a00                    push    0

00bd1158 6a00                    push    0

00bd115a 8b55ec                mov     edx,dword ptr [ebp-14h]

00bd115d 8b04950070bf00  mov     eax,dword ptr FaultyApp!pFnarry (00bf7000)[edx*4]   

This above sections seems to be initializing a function pointer array for thread functions.  We know this because of the call to CreateThread API a few lines below....

                                

00bd1164 50              push    eax

00bd1165 6a00            push    0

00bd1167 6a00            push    0

00bd1169 ff154cb0bf00    call    dword ptr [FaultyApp!_imp__CreateThread (00bfb04c)]

(the reader is encouraged to use MSDN to see the signature for CreateThread and all the parameters pushed into it inside this function to decrypt what all was passed)

00bd116f 8b4dec          mov     ecx,dword ptr [ebp-14h]

00bd1172 89448df0       mov     dword ptr [ebp+ecx*4-10h],eax

00bd1176 8b55ec          mov     edx,dword ptr [ebp-14h]

00bd1179 837c95f000    cmp     dword ptr [ebp+edx*4-10h],0

00bd117e 7505              jne     FaultyApp!main+0x65 (00bd1185)

FaultyApp!main+0x60:

00bd1180 83c8ff          or      eax,0FFFFFFFFh

00bd1183 eb32            jmp     FaultyApp!main+0x97 (00bd11b7)

FaultyApp!main+0x65:

00bd1185 ebbe            jmp     FaultyApp!main+0x25 (00bd1145)

FaultyApp!main+0x67:

00bd1187 a1b892bf00      mov     eax,dword ptr [FaultyApp!hStartThreads (00bf92b8)]

00bd118c 50                    push    eax

00bd118d ff1540b0bf00   call    dword ptr [FaultyApp!_imp__SetEvent (00bfb040)]

00bd1193 6a00                push    0

00bd1195 68f4010000      push    1F4h

00bd119a ff1544b0bf00   call    dword ptr [FaultyApp!_imp__SleepEx (00bfb044)]

00bd11a0 e885010000     call    FaultyApp!DllFunction (00bd132a)                      

After setting an event and sleeping for a while, it finally jumps to this function in the above line. DllFunction. We need to disassemble it to see where it leads us. I am not pasting the rest of the disassembly of the Main function as it is irrelevant to the rest of the context.

Lets start with the disassembly of DllFunction.

0:004> u FaultyApp!DllFunction

FaultyApp!DllFunction:

00bd132a ff2504b0bf00    jmp     dword ptr [FaultyApp!_imp__DllFunction (00bfb004)]

FaultyApp!__security_check_cookie:

00bd1330 3b0d1070bf00    cmp     ecx,dword ptr [FaultyApp!__security_cookie (00bf7010)]

00bd1336 7502                  jne     FaultyApp!__security_check_cookie+0xa (00bd133a)

00bd1338 f3c3                  rep ret

00bd133a e908020000       jmp     FaultyApp!__report_gsfailure (00bd1547)

FaultyApp!__tmainCRTStartup:

00bd133f 6a14                  push    14h

00bd1341 68d05dbf00       push    offset FaultyApp!__rtc_tzz+0x104 (00bf5dd0)

00bd1346 e8551a0000       call    FaultyApp!__SEH_prolog4 (00bd2da0)

Whats going on? This doesn't look like a function at all....so look closely at the first line copied again in the line below for your convenience...

00bd132a ff2504b0bf00    jmp     dword ptr [FaultyApp!_imp__DllFunction (00bfb004)]

This is actually a function pointer which is being de-referenced. So if it is a pointer, we need to see what it is pointing at:

0:004> dd 00bfb004

00bfb004  6ce11005 00000000 00000000 00000000

00bfb014  00000000 00000000 00000000 00000000

00bfb024  00000000 00000000 00000000 00000000

00bfb034  77d72c8a 77d622c0 77d62280 76e71691

00bfb044  76e71215 76e7324c 76e73495 76e741e8

00bfb054  76e75169 76e74a25 76e751fd 76e711c0

00bfb064  76e711a9 76e717b8 76e71420 77d8107b

00bfb074  77d79dd5 76e779d8 76e74a37 76e71222

Lets have a look at what is actually at this address.

0:004> u 6ce11005

FaultyDLL!ILT+0(_DllFunction):

6ce11005 e916000000      jmp     FaultyDLL!DllFunction (6ce11020)

FaultyDLL!ILT+5(_DllMain:

6ce1100a e931000000      jmp     FaultyDLL!DllMain (6ce11040)

6ce1100f cc              int     3

6ce11010 cc              int     3

6ce11011 cc              int     3

6ce11012 cc              int     3

6ce11013 cc              int     3

6ce11014 cc              int     3

It shows the name of a different module. FaultyDll. Looks like it is a DLL function being used, through the import address table (I know the name of the module has the word DLL in it, but that is not why i reached this conclusion).

Now lets disassemble the actual function the address of which we have been able to decipher.

0:004> u 6ce11020

FaultyDLL!DllFunction:

6ce11020 55                    push    ebp

6ce11021 8bec                mov     ebp,esp

6ce11023 68e092e36c      push    offset FaultyDLL!g_CritSec (6ce392e0)

6ce11028 ff1500b0e36c   call    dword ptr [FaultyDLL!_imp__EnterCriticalSection (6ce3b000)]

6ce1102e b801000000      mov     eax,1

6ce11033 5d                    pop     ebp

6ce11034 c3                    ret

6ce11035 cc                    int     3

It is a small function and what do we see in it? It enter's the critical section, but never leaves it. This is the bug!!

Main is shown as the owner of the critical section because it calls a faulty function which enters but never leaves the critical section.

Note: This is a very simple test application tailor made to produce the hang, real life scenarios might be much harder to debug and locate. But I do hope that this gives a starting point to folks wanting to chase such bugs!

Trivia : !cs command also gives info about all critical sections. !cs <address of crit> will dump info about that particular critical section. I deliberately avoided using it, as this exercise helps one understand such debugging scenarios in depth, rather than being dependent on a debugger extension.

WinDbg : An Introduction To Windows Heaps

WinDbg : An Introduction To Windows Heaps 

1. dt
2. dd
3. !process
4. .process
5. $ 

Heaps are used by applications which need to allocate and release memory dynamically. Even though the heap is the most common facility to accommodate dynamic memory allocations, it is not the only way. Several other methods are available:

• Memory can be requested through the C run time (CRT).
• The virtual memory manager or some other private memory manager.

Even though these methods are different, they are tightly coupled internally at the memory manager's level.

When a process starts, the heap manager automatically creates a new heap called the 'default process heap'. Although some processes are known to use the default heap, a very very large number rely on the CRT heap (using malloc/free family of APIs, or the default new/delete family of APIs).

Some processes however also create additional heaps (via HeapCreate(...) ). this method has advantage of isolating different components running in the process. Thus memory leaks can be easier to detect and debug, if we know exactly which heap is leaking. It is indeed good programming practice. Kernel programmers are used to using different tags for allocating memory for different objects, this method described above is the usermode solution to the problem.

The Windows heap manager has two sub components. 

• Front End Allocator.
• Back End Allocator.

The Front End Allocator is an optimization layer for the actual Back End Allocator. Thus, by choosing different front end allocators applications with differnt memory requirements can function appropriately. Example, some programs might expect small bursts of allocations, and thus might prefer to use a low fragmentation front end allocator. 

Windows supplies two different front end allocators:

• Low fragmentation (LF) Front End Allocator.
• Look aside List (LAL) Front End Allocator.

Vista and above use the LF allocator by default, where as all older generations use the LAL allocator. The differences between these allocators re beyond the scope of the current discussion.

If the Front End Allocator is unable to satisfy the allocation requests, then the Back End Allocator takes over. It is made up of free lists. Each list has blocks of specific sizes. Such free blocks are sorted in ascending order of sizes. Again, a more elaborate discussion on how this is managed, or of it's algorithms is beyond the scope of this current article.

Enough theory, lets get into some practicals.

The first step for us is to determine which all heaps are active for a process.

kd> $ Lets use the lsass process as an example.

kd> $ First we need to find it's EPROCESS

kd> !process 0 0 lsass.exe

PROCESS 84cdc860  SessionId: 0  Cid: 0208    Peb: 7ffda000  ParentCid: 018c

    DirBase: 1eed30e0  ObjectTable: 95f00d18  HandleCount: 513.

    Image: lsass.exe

kd> $ Next we need to switch contexts to this process (this is because this session is running from Kernel mode)

kd> .process /p /r 84cdc860  

Implicit process is now 84cdc860

Loading User Symbols

............................................................

kd> $ The Process execution block (_PEB) helps us with finding the active heaps, so lets use dt to find it

kd> dt _PEB @$peb

ntdll!_PEB

   +0x000 InheritedAddressSpace : 0 ''

   +0x001 ReadImageFileExecOptions : 0 ''

   +0x002 BeingDebugged    : 0 ''

   +0x003 BitField         : 0x8 ''

   +0x003 ImageUsesLargePages : 0y0

   +0x003 IsProtectedProcess : 0y0

   +0x003 IsLegacyProcess  : 0y0

   +0x003 IsImageDynamicallyRelocated : 0y1

   +0x003 SkipPatchingUser32Forwarders : 0y0

   +0x003 SpareBits        : 0y000

   +0x004 Mutant           : 0xffffffff Void

   +0x008 ImageBaseAddress : 0x00d90000 Void

   +0x00c Ldr              : 0x77c77880 _PEB_LDR_DATA

   +0x010 ProcessParameters : 0x00330f18 _RTL_USER_PROCESS_PARAMETERS

   +0x014 SubSystemData    : (null) 

   +0x018 ProcessHeap      : 0x00330000 Void

   +0x01c FastPebLock      : 0x77c77380 _RTL_CRITICAL_SECTION

   +0x020 AtlThunkSListPtr : (null) 

   +0x024 IFEOKey          : (null) 

   +0x028 CrossProcessFlags : 0

   +0x028 ProcessInJob     : 0y0

   +0x028 ProcessInitializing : 0y0

   +0x028 ProcessUsingVEH  : 0y0

   +0x028 ProcessUsingVCH  : 0y0

   +0x028 ProcessUsingFTH  : 0y0

   +0x028 ReservedBits0    : 0y000000000000000000000000000 (0)

   +0x02c KernelCallbackTable : 0x7790d568 Void

   +0x02c UserSharedInfoPtr : 0x7790d568 Void

   +0x030 SystemReserved   : [1] 0

   +0x034 AtlThunkSListPtr32 : 0

   +0x038 ApiSetMap        : 0x77de0000 Void

   +0x03c TlsExpansionCounter : 0

   +0x040 TlsBitmap        : 0x77c77260 Void

   +0x044 TlsBitmapBits    : [2] 0x1fffff

   +0x04c ReadOnlySharedMemoryBase : 0x7f6f0000 Void

   +0x050 HotpatchInformation : (null) 

   +0x054 ReadOnlyStaticServerData : 0x7f6f0590  -> (null) 

   +0x058 AnsiCodePageData : 0x7ffb0000 Void

   +0x05c OemCodePageData  : 0x7ffc0224 Void

   +0x060 UnicodeCaseTableData : 0x7ffd0648 Void

   +0x064 NumberOfProcessors : 1

   +0x068 NtGlobalFlag     : 0

   +0x070 CriticalSectionTimeout : _LARGE_INTEGER 0xffffe86d`079b8000

   +0x078 HeapSegmentReserve : 0x100000

   +0x07c HeapSegmentCommit : 0x2000

   +0x080 HeapDeCommitTotalFreeThreshold : 0x10000

   +0x084 HeapDeCommitFreeBlockThreshold : 0x1000

   +0x088 NumberOfHeaps    : 4

   +0x08c MaximumNumberOfHeaps : 0x10

   +0x090 ProcessHeaps     : 0x77c77500  -> 0x00330000 Void

   +0x094 GdiSharedHandleTable : 0x004d0000 Void

   +0x098 ProcessStarterHelper : (null) 

   +0x09c GdiDCAttributeList : 0x14

   +0x0a0 LoaderLock       : 0x77c77340 _RTL_CRITICAL_SECTION

   +0x0a4 OSMajorVersion   : 6

   +0x0a8 OSMinorVersion   : 1

   +0x0ac OSBuildNumber    : 0x1db1

   +0x0ae OSCSDVersion     : 0x100

   +0x0b0 OSPlatformId     : 2

   +0x0b4 ImageSubsystem   : 2

   +0x0b8 ImageSubsystemMajorVersion : 6

   +0x0bc ImageSubsystemMinorVersion : 1

   +0x0c0 ActiveProcessAffinityMask : 1

   +0x0c4 GdiHandleBuffer  : [34] 0

   +0x14c PostProcessInitRoutine : (null) 

   +0x150 TlsExpansionBitmap : 0x77c77268 Void

   +0x154 TlsExpansionBitmapBits : [32] 1

   +0x1d4 SessionId        : 0

   +0x1d8 AppCompatFlags   : _ULARGE_INTEGER 0x0

   +0x1e0 AppCompatFlagsUser : _ULARGE_INTEGER 0x0

   +0x1e8 pShimData        : (null) 

   +0x1ec AppCompatInfo    : (null) 

   +0x1f0 CSDVersion       : _UNICODE_STRING "Service Pack 1"

   +0x1f8 ActivationContextData : 0x00040000 _ACTIVATION_CONTEXT_DATA

   +0x1fc ProcessAssemblyStorageMap : (null) 

   +0x200 SystemDefaultActivationContextData : 0x00030000 _ACTIVATION_CONTEXT_DATA

   +0x204 SystemAssemblyStorageMap : (null) 

   +0x208 MinimumStackCommit : 0

   +0x20c FlsCallback      : 0x0033daa0 _FLS_CALLBACK_INFO

   +0x210 FlsListHead      : _LIST_ENTRY [ 0x347828 - 0x3cb680 ]

   +0x218 FlsBitmap        : 0x77c77270 Void

   +0x21c FlsBitmapBits    : [4] 7

   +0x22c FlsHighIndex     : 2

   +0x230 WerRegistrationData : (null) 

   +0x234 WerShipAssertPtr : (null) 

   +0x238 pContextData     : 0x00050000 Void

   +0x23c pImageHeaderHash : (null) 

   +0x240 TracingFlags     : 0

   +0x240 HeapTracingEnabled : 0y0

   +0x240 CritSecTracingEnabled : 0y0

   +0x240 SpareTracingBits : 0y000000000000000000000000000000 (0)

kd> $ Now lets use dd to see what this address contains

kd> dd 0x77c77500  

77c77500  00330000 00010000 00250000 00d10000

77c77510  00000000 00000000 00000000 00000000

77c77520  00000000 00000000 00000000 00000000

77c77530  00000000 00000000 00000000 00000000

77c77540  00000000 77c77340 77c7ab08 77c77220

77c77550  00000000 00000004 00000000 00000000

77c77560  77c77220 003e3198 00000000 00000000

77c77570  00000000 00000000 00000000 00000000

The Default Process heap pointer is always the first one in this list. Since most applications work with the default heap, we will focus our attention on that.

The _HEAP structure in Windows is used to maintain a heap. So lets typecast this address to it.

kd> dt _HEAP 0x00330000

ntdll!_HEAP

   +0x000 Entry            : _HEAP_ENTRY

   +0x008 SegmentSignature : 0xffeeffee

   +0x00c SegmentFlags     : 0

   +0x010 SegmentListEntry : _LIST_ENTRY [ 0x3300a8 - 0x3300a8 ]

   +0x018 Heap             : 0x00330000 _HEAP

   +0x01c BaseAddress      : 0x00330000 Void

   +0x020 NumberOfPages    : 0x100

   +0x024 FirstEntry       : 0x00330588 _HEAP_ENTRY

   +0x028 LastValidEntry   : 0x00430000 _HEAP_ENTRY

   +0x02c NumberOfUnCommittedPages : 0x47

   +0x030 NumberOfUnCommittedRanges : 1

   +0x034 SegmentAllocatorBackTraceIndex : 0

   +0x036 Reserved         : 0

   +0x038 UCRSegmentList   : _LIST_ENTRY [ 0x3e8ff0 - 0x3e8ff0 ]

   +0x040 Flags            : 2

   +0x044 ForceFlags       : 0

   +0x048 CompatibilityFlags : 0

   +0x04c EncodeFlagMask   : 0x100000

   +0x050 Encoding         : _HEAP_ENTRY

   +0x058 PointerKey       : 0x37d910ba

   +0x05c Interceptor      : 0

   +0x060 VirtualMemoryThreshold : 0xfe00

   +0x064 Signature        : 0xeeffeeff

   +0x068 SegmentReserve   : 0x100000

   +0x06c SegmentCommit    : 0x2000

   +0x070 DeCommitFreeBlockThreshold : 0x800

   +0x074 DeCommitTotalFreeThreshold : 0x2000

   +0x078 TotalFreeSize    : 0xac0

   +0x07c MaximumAllocationSize : 0x7ffdefff

   +0x080 ProcessHeapsListIndex : 1

   +0x082 HeaderValidateLength : 0x138

   +0x084 HeaderValidateCopy : (null) 

   +0x088 NextAvailableTagIndex : 0

   +0x08a MaximumTagIndex  : 0

   +0x08c TagEntries       : (null) 

   +0x090 UCRList          : _LIST_ENTRY [ 0x3e8fe8 - 0x3e8fe8 ]

   +0x098 AlignRound       : 0xf

   +0x09c AlignMask        : 0xfffffff8

   +0x0a0 VirtualAllocdBlocks : _LIST_ENTRY [ 0x3300a0 - 0x3300a0 ]

   +0x0a8 SegmentList      : _LIST_ENTRY [ 0x330010 - 0x330010 ]

   +0x0b0 AllocatorBackTraceIndex : 0

   +0x0b4 NonDedicatedListLength : 0

   +0x0b8 BlocksIndex      : 0x00330150 Void

   +0x0bc UCRIndex         : 0x00330590 Void

   +0x0c0 PseudoTagEntries : (null) 

   +0x0c4 FreeLists        : _LIST_ENTRY [ 0x3db6a8 - 0x3de378 ]

   +0x0cc LockVariable     : 0x00330138 _HEAP_LOCK

   +0x0d0 CommitRoutine    : 0x37d910ba     long  +37d910ba

   +0x0d4 FrontEndHeap     : 0x00336548 Void

   +0x0d8 FrontHeapLockCount : 0

   +0x0da FrontEndHeapType : 0x2 ''

   +0x0dc Counters         : _HEAP_COUNTERS

   +0x130 TuningParameters : _HEAP_TUNING_PARAMETERS

Note: The _HEAP structure might be different for different versions of Windows.

A point to note here is that inside the _PEB there are two fields:

kd> dt _PEB ProcessH*

ntdll!_PEB

   +0x018 ProcessHeap : Ptr32 Void

   +0x090 ProcessHeaps : Ptr32 Ptr32 Void

The ProcessHeaps is an array of pointers to the various heaps for the process. The ProcessHeap on the other hand will always store the address of the default heap of the process.

To browse all heaps in this process, we can use the WinDbg .for command, which is described here.

kd> .for (r $t0 = 0; @$t0 < 0x4; r $t0 = @$t0 + 1){dt _HEAP poi(0x77c77500 + ((@$t0)*4)) - y SegmentSignature}

ntdll!_HEAP
   +0x008 SegmentSignature : 0xffeeffee
ntdll!_HEAP
   +0x008 SegmentSignature : 0xffeeffee
ntdll!_HEAP
   +0x008 SegmentSignature : 0xffeeffee
ntdll!_HEAP
   +0x008 SegmentSignature : 0xffeeffee

We will revisit heaps again for further elaborate discussions and even walking the heaps when we explore heap corruption scenarios.

WinDbg : Trying To Find The Import Address Table (IAT) Of A Binary

WinDbg : Trying To Find The Import Address Table (IAT) Of A Binary

In the last article we learnt how to use the basic WinDbg commands we had learnt, to find out useful information within the loaded binary images. We will extend that study here, to see how other relevant fields output by the !dh command can be used. The focus of this article would be to use the WinDbg commands and to find the Import Address Table of the loaded image and to see the list of functions that it has imported.

Commands used in this article are:

1. !dh
2. dds
3. ln

For this example I picked the DLL called KernelBase. you can use any other DLL or executable of your choice.I also used the -f switch to trim the output of !dh to the relevant sections we need here.

The IAT is an array of pointers which are loaded by the PE Image Loader. The IAT is used primarily as form of a lookup table, which is used to call function present in other library modules (.DLLs). Since the executable module will not the the know memory addresses of the libraries and it's stored functions, it brings the in the purpose of the IAT. The IAT slots will be written with memory addresses by the linker. 

The IAT is part of a larger data structure called the _IMAGE_IMPORT_DESCRIPTOR, which also contains another lookup table called the INT (Import Name Table), which is identical to the IAT.

Unfortunately, this structure is not exported in WinDbg, so we cannot use the dt command to see it. However, it is available in WinNt.H, so I will copy paste it for our convenience.

typedef struct _IMAGE_IMPORT_DESCRIPTOR {

    union {

        DWORD   Characteristics;            // 0 for terminating null import descriptor

        DWORD   OriginalFirstThunk;         // RVA to original unbound IAT (PIMAGE_THUNK_DATA)

    } DUMMYUNIONNAME;

    DWORD   TimeDateStamp;                  // 0 if not bound,

                                            // -1 if bound, and real date\time stamp

                                            //     in IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT (new BIND)

                                            // O.W. date/time stamp of DLL bound to (Old BIND)

    DWORD   ForwarderChain;                 // -1 if no forwarders

    DWORD   Name;

    DWORD   FirstThunk;                     // RVA to IAT (if bound this IAT has actual addresses)

} IMAGE_IMPORT_DESCRIPTOR;

The OriginalFirstThunk points to the Import Name Table and the FirstThunk points to the Import Address Table. We can find these in WinDbg, but we would need to use the !dh extension to do so.

kd> !dh kernelbase.dll -f

File Type: DLL

FILE HEADER VALUES

     14C machine (i386)

       4 number of sections

4CE7B8F0 time date stamp Sat Nov 20 17:32:56 2010

       0 file pointer to symbol table

       0 number of symbols

      E0 size of optional header

    2102 characteristics

            Executable

            32 bit word machine

            DLL

OPTIONAL HEADER VALUES

     10B magic #

    9.00 linker version

   42400 size of code

    3E00 size of initialized data

       0 size of uninitialized data

    7DE0 address of entry point

    1000 base of code

         ----- new -----

75d70000 image base

    1000 section alignment

     200 file alignment

       3 subsystem (Windows CUI)

    6.01 operating system version

    6.01 image version

    6.01 subsystem version

   4A000 size of image

     400 size of headers

   4984C checksum

00040000 size of stack reserve

00001000 size of stack commit

00100000 size of heap reserve

00001000 size of heap commit

     140  DLL characteristics

            Dynamic base

            NX compatible

    18D4 [    4E67] address [size] of Export Directory

   406A8 [       28] address [size] of Import Directory

   46000 [      530] address [size] of Resource Directory

          0 [         0] address [size] of Exception Directory

          0 [         0] address [size] of Security Directory

   47000 [   25D0] address [size] of Base Relocation Directory

   43294 [       38] address [size] of Debug Directory

         0 [          0] address [size] of Description Directory

         0 [          0] address [size] of Special Directory

         0 [          0] address [size] of Thread Storage Directory

 1E110 [         40] address [size] of Load Configuration Directory

     278 [        1C] address [size] of Bound Import Directory

   1000 [       65C] address [size] of Import Address Table Directory

        0 [           0] address [size] of Delay Import Directory

        0 [           0] address [size] of COR20 Header Directory

        0 [           0] address [size] of Reserved Directory

Using the dd command to dump the memory addresses of the Import Directory, we can see the relevant members mentioned earlier.

kd> dd 406A8+kernelbase
75db06a8  000406dc ffffffff ffffffff 000406d0
75db06b8  00001000 00000000 00000000 00000000
75db06c8  00000000 00000000 6c64746e 6c642e6c
75db06d8  9090006c 00040d38 00040d58 00040d78
75db06e8  00040d86 00040d90 00040da4 00040db2
75db06f8  00040dce 00040dee 00040e00 00040e12
75db0708  00040e2e 00040e42 00040e5e 00040e78

75db0718  00040e82 00040e9a 00040eb2 00040ec6

000406dc is the address of the OriginalFirstThunk and the 00001000 is the address of the FirstThunk. 

Lets use the dd command to dump the OriginalFirstThunk.

kd> dd 000406dc+kernelbase
75db06dc  00040d38 00040d58 00040d78 00040d86
75db06ec  00040d90 00040da4 00040db2 00040dce
75db06fc  00040dee 00040e00 00040e12 00040e2e
75db070c  00040e42 00040e5e 00040e78 00040e82
75db071c  00040e9a 00040eb2 00040ec6 00040ede
75db072c  00040ef2 00040f0a 00040f36 00040f50
75db073c  00040f7a 00040f92 00040fa0 00040fb8

75db074c  00040fca 00040fda 00040ff4 00040ffe

The addresses shown are for the INT, and by using the dc command on one of the selected offsets with the base address, you will be able to view the name of the function. 

kd> dc 00040d38+kernelbase
75db0d38  745204c2 696e556c 65646f63 69727453  ..RtlUnicodeStri
75db0d48  6f54676e 69736e41 69727453 0000676e  ngToAnsiString..
75db0d58  74520268 736e416c 72745369 54676e69  h.RtlAnsiStringT
75db0d68  696e556f 65646f63 69727453 0000676e  oUnicodeString..
75db0d78  765f074c 70776e73 746e6972 077e0066  L._vsnwprintf.~.
75db0d88  736d656d 00007465 74520346 6572466c  memset..F.RtlFre
75db0d98  736e4165 72745369 00676e69 74520348  eAnsiString.H.Rt

75db0da8  6572466c 61654865 02da0070 446c7452  lFreeHeap...RtlD

We can display similar information with the FirstThunk, using FirstThunk can be a bit cleaner, since we can use the ln command with the address to find the exact function, which will be displayed cleanly with WinDbg.

kd> dd 00001000+kernelbase
75d71000  77bf9e8e 77bf572f 77c0caaa 77bd5340
75d71010  77bf3306 77bf2c6a 77bf9ac5 77bfa149
75d71020  77bf2dd6 77c3ea50 77c08b1b 77c07dfd
75d71030  77be6048 77be6678 77be54c8 77be6638
75d71040  77be55d8 77be66f8 77bf30fb 77bf7e74
75d71050  77be6398 77bf3063 77be63a8 77bf2fed
75d71060  77be5658 77be5cd8 77be6a18 77be5a08
75d71070  77be55a8 77be6018 77bd4500 77c0314e
kd> ln 77bf9e8e
(77bf9e8e)   ntdll!RtlUnicodeStringToAnsiString   |  (77bf9f5b)   ntdll!RtlpScanEnvironment
Exact matches:

    ntdll!RtlUnicodeStringToAnsiString (<no parameter info>)

Now, going back to the original output of the !dh, we see that the offset 0x1000 is also shown as:

1000 [       65C] address [size] of Import Address Table Directory

So this extension does give us the FirstThunk directly.

We can use the dds command to dump address range from the virtual address of the beginning of the IAT array. dds is equivalent to dd except that 's' stands for resolving symbols (if available). This means that if any of the addresses dumped in word format resolves to a known symbol, then the corresponding name would also be displayed. (Also read up on the dps command to see how it can also be used in this case).

kd> dds kernelbase + 1000 

75d71000  77bf9e8e ntdll!RtlUnicodeStringToAnsiString

75d71004  77bf572f ntdll!RtlAnsiStringToUnicodeString

75d71008  77c0caaa ntdll!_vsnwprintf

75d7100c  77bd5340 ntdll!memset

75d71010  77bf3306 ntdll!RtlFreeAnsiString

75d71014  77bf2c6a ntdll!RtlFreeHeap

75d71018  77bf9ac5 ntdll!RtlDeleteCriticalSection

75d7101c  77bfa149 ntdll!RtlInitializeCriticalSection

75d71020  77bf2dd6 ntdll!RtlAllocateHeap

75d71024  77c3ea50 ntdll!CsrVerifyRegion

75d71028  77c08b1b ntdll!CsrClientConnectToServer

75d7102c  77c07dfd ntdll!RtlCreateTagHeap

75d71030  77be6048 ntdll!ZwQueryInformationProcess

75d71034  77be6678 ntdll!NtSetInformationProcess

75d71038  77be54c8 ntdll!NtClose

75d7103c  77be6638 ntdll!ZwSetInformationFile

75d71040  77be55d8 ntdll!NtCreateIoCompletion

75d71044  77be66f8 ntdll!NtSetIoCompletion

75d71048  77bf30fb ntdll!RtlSetLastWin32Error

75d7104c  77bf7e74 ntdll!SbSelectProcedure

75d71050  77be6398 ntdll!NtRemoveIoCompletion

75d71054  77bf3063 ntdll!RtlDeactivateActivationContextUnsafeFast

75d71058  77be63a8 ntdll!NtRemoveIoCompletionEx

75d7105c  77bf2fed ntdll!RtlActivateActivationContextUnsafeFast

75d71060  77be5658 ntdll!ZwCreateNamedPipeFile

75d71064  77be5cd8 ntdll!NtOpenFile

75d71068  77be6a18 ntdll!ZwWaitForSingleObject

75d7106c  77be5a08 ntdll!ZwFsControlFile

75d71070  77be55a8 ntdll!ZwCreateEvent

75d71074  77be6018 ntdll!ZwQueryInformationFile

75d71078  77bd4500 ntdll!_allmul

75d7107c  77c0314e ntdll!RtlSetDaclSecurityDescriptor

So we see that we are getting the same output which ever method we prefer to use.

Try out these command sequences for other binary images to see the outputs. Beware, not all binaries import, though that would be an extremely rare occurrence. Also please do note, that in the above example I ignored the size of the import table 0x65C. The right way to dump would be to ask dds to only dump till the end of the import address table, else it might show garbage.