In this topic, we are going to discuss about memory dump collection techniques from field experience.
ProcDump.exe is a very helpful tool that monitors the process and helps in grabbing memory dumps.
Monitor for first chance exceptions
C:\Temp>procdump.exe –e 1 –f “” ProcessNameOrProcessId
C:\Temp>procdump.exe -e 1 -f "" 6532
ProcDump v7.1 - Writes process dump files
Copyright (C) 2009-2014 Mark Russinovich
Sysinternals - www.sysinternals.com
With contributions from Andrew Richards
Process: w3wp.exe (6532)
CPU threshold: n/a
Performance counter: n/a
Commit threshold: n/a
Threshold seconds: 10
Hung window check: Disabled
Log debug strings: Disabled
Exception monitor: First Chance+Unhandled
Exception filter: Display Only
Terminate monitor: Disabled
Cloning type: Disabled
Concurrent limit: n/a
Avoid outage: n/a
Number of dumps: 1
Dump folder: C:\Temp\
Dump filename/mask: PROCESSNAME_YYMMDD_HHMMSS
Press Ctrl-C to end monitoring without terminating the process.
CLR Version: v4.0.30319
[21:47:51] Exception: E0434F4D.System.IO.IOException ("The user name or password
is incorrect.
")
[21:47:51] Exception: E0434F4D.System.IO.IOException ("The user name or password
is incorrect.
")
[21:47:51] Exception: E0434F4D.System.IO.IOException ("The user name or password
is incorrect.
")
[21:47:51] Exception: E0434F4D.System.IO.IOException ("The user name or password
is incorrect.
")
[21:48:01] Exception: E0434F4D.System.TimeoutException ("The service's security
session did not receive a 'close' message from the client within the configured
timeout (00:00:10).")
^C
[21:49:39] Dump count not reached.
Take a dump on the exception
1. Pull the exception code or name
2. Provide the above in command prompt like the following:
c:\temp>procdump.exe -ma -e 1 -f "System.IO.IOException" ProcessNameOrId
Launch the process and monitor for exceptions
1. Put the ProcDump.exe in the application exe folder location (*** helpful in case of stand-alone application)
2. Set the following command prompt option
c:\MyAppliacation>procdump.exe -e 1 -f "" -x C:\MyApplication TestApplication.exe
ProcDump v7.1 - Writes process dump files
Copyright (C) 2009-2014 Mark Russinovich
Sysinternals - www.sysinternals.com
With contributions from Andrew Richards
Process: TestApplication.exe (5356)
CPU threshold: n/a
Performance counter: n/a
Commit threshold: n/a
Threshold seconds: 10
Hung window check: Disabled
Log debug strings: Disabled
Exception monitor: First Chance+Unhandled
Exception filter: Display Only
Terminate monitor: Disabled
Cloning type: Disabled
Concurrent limit: n/a
Avoid outage: n/a
Number of dumps: 1
Dump folder: C:\MyApplication\
Dump filename/mask: PROCESSNAME_YYMMDD_HHMMSS
Press Ctrl-C to end monitoring without terminating the process.
CLR Version: v4.0.30319
[15:33:22] Exception: E0434F4D.System.Net.Sockets.SocketException ("No such host
is known")
[15:33:22] Exception: E0434F4D.System.Net.Sockets.SocketException ("No such host
is known")
[15:33:22] Exception: E0434F4D.System.Reflection.TargetInvocationException ("Exc
eption has been thrown by the target of an invocation.")
[15:33:22] Exception: E0434F4D.System.Net.Sockets.SocketException ("No such host
is known")
[15:33:32] The process has exited.
[15:33:32] Dump count not reached.
Take a crash dump with application launched
1. From the above list, we can identify the exception type.
2. Set a rule like the following:
c:\MyApplication>procdump.exe -ma -e 1 -f "System.Net.Sockets.SocketException" -x C:\MyApplication TestApplication.exe
ProcDump v7.1 - Writes process dump files
Copyright (C) 2009-2014 Mark Russinovich
Sysinternals - www.sysinternals.com
With contributions from Andrew Richards
Process: TestApplication.exe (11180)
CPU threshold: n/a
Performance counter: n/a
Commit threshold: n/a
Threshold seconds: 10
Hung window check: Disabled
Log debug strings: Disabled
Exception monitor: First Chance+Unhandled
Exception filter: *System.Net.Sockets.SocketException*
Terminate monitor: Disabled
Cloning type: Disabled
Concurrent limit: n/a
Avoid outage: n/a
Number of dumps: 1
Dump folder: C:\MyApplication\
Dump filename/mask: PROCESSNAME_YYMMDD_HHMMSS
Press Ctrl-C to end monitoring without terminating the process.
CLR Version: v4.0.30319
[15:35:05] Exception: E0434F4D.System.Net.Sockets.SocketException ("No such host
is known")
[15:35:05] Dump 1 initiated: C:\MyApplication\TestApplication.exe_160420_153505.dmp
[15:35:05] Dump 1 writing: Estimated dump file size is 150 MB.
[15:35:06] Dump 1 complete: 150 MB written in 1.0 seconds
[15:35:06] Dump count reached.
Wait for the specified process
1. Using -w option in procdump means, it will wait for the particular process to be running in system.
2. -w option can also be associated with -e and -f options for crash scenarios.
C:\temp>procdump.exe -w w3wp.exe ProcDump v7.1 - Writes process dump files Copyright (C) 2009-2014 Mark Russinovich Sysinternals - www.sysinternals.com With contributions from Andrew Richards Waiting for process named w3wp.exe... [03:01:48] Dump 1 initiated: C:\temp\w3wp.exe_160423_030148.dmp [03:01:55] Dump 1 complete: 52 MB written in 6.9 seconds [03:01:55] Dump count reached.
* The same can be applied for a stand-alone exe as well.
* If we have multiple application pools running (i.e. multiple w3wp process IDs) and we have identified the w3wp process ID, then procDump tool can help.
* If w3wp process ID is not identified and we have multiple w3wp, it is better to go with DebugDiag where it gives us control to attach with an application pool directly. Runtime process ID binding is taken care by this approach.
High CPU scenario
Write up to 3 mini dumps of a process named ‘MyApplication’ when it exceeds 95% CPU usage for five seconds:
C:\Temp>procdump -ma -c 95 -s 5 -n 3 MyApplication.exe
* Benefit over here is that when the memory dump is opened in WinDbg or any other debugger, we can directly view the probable culprit thread among the list of threads.
For more details, please visit the ProcDump documentation.