[Debug]Find the Cause of Web Site Hang by Analyzing Dump File

1. Download Debugging Tools for Windows

http://www.microsoft.com/whdc/devtools/debugging/default.mspx
Then, Install Debugging Tools

2.Conifgure Symbols

Without Symbols, dump file is just some meaningless and useless things. There are two choices to use the Symbols file: 1. download the whole symbol install files. 2. Download certain file when we use it. It is time consume to download a 300Mb file, so I choose the second option:
Open WinDbg, File->Symbol Search Path, next, fill the text box with:
      srv*c:\symbols\public*http://msdl.microsoft.com/download/symbols

3. Start a Test Web Site

Get a buggy web site here. We are going to use TinyGet to simulate some http request. TinyGet is one tool of IIS 6.0 Resource Kit Tools
TinyGet example:
    tinyget -srv:xhinker.com -uri:/default.aspx -h
-h means return http response head. if you want to output all reponse info, use -t

4. Get hang dump file

Start adplus, type the following command, but don't hit "enter" key
     adplus –hang –pn w3wp.exe –quiet
-quiet means you won't get any unnecessary popup boxes.
Start Tiny get, type the following command and hit enter.
     tinyget -srv:localhost -uri:/BuggyBits/FeaturedProducts.aspx -threads:30 -loop:50
switch to adplus script command window, hit enter. after several seconds you will find the hang web site's dump file in C:\Program Files\Debugging Tools for Windows (x86)\Hang_Mode__Date_09-26-2009__Time_14-03-45PM (It may diff in your computer).

5.Analyse the dump file

Start WinDbg, load the dump file just created(File->Open Crash Dump)

Load SOS(copy sos.dll from C:\Windows\Microsoft.NET\Framework\v2.0.50727 to C:\Program Files\Debugging Tools for Windows (x86) before running the following command)
    .load sos
(the following words are mainly come from http://blogs.msdn.com/tess/archive/2008/02/04/net-debugging-demos-lab-1-hang.aspx)
Examine the stacks
1. Examine the native callstacks
    ~* kb 2000
2. Examine the .net callstacks
    ~* e !clrstack
Troubleshoot the hang
1. Determine the ID of the thread owning the lock
    !syncblk

Q: What thread owns the lock?
A:Thread 23 owns the lock
Q: How many threads are waiting for the lock?
Hint: MonitorHeld = 1 for each owner and 2 for each waiter.
A:(17-1)/2=8 threads are waiting for the lock.
2. Pick one of the waiters (Hint: waiters will sit in AwareLock::Enter) and take a look at what it is doing.
    ~5s                          (move to thread 5, replace 5 with actual thread ID)
    kb 2000                    (examine native stack)
    !clrstack                    (examine .net stack)
3. Determine what the owning thread is doing 
    ~5s                          (move to thread 5, replace 5 with actual thread ID)
    kb 2000                    (examine native stack)
    !clrstack                    (examine .net stack)

Q: Why is it blocking?
A:Thread.Sleep() makes the web site hanging there.
So, it is interesting, right?

blog comments powered by Disqus