A web api in production was leaking memory, lots of it. It was consuming over 27 gigs of memory where the usual consumption is around 200 Mbs.
I used this approach to pinpoint what was leaking since the memory leak was replicated again on a staging environment which I had access to. I needed to act quickly before the process chokes the server and becomes in need of a reboot.
- Create a process dump file. This can be done through task manager. The task manager you get from ctr+alt+del is a 64 bit version, this will generate a 64 bit dump. To generate a 32 bit dump use the 32 bit task manager (C:\Windows\SysWOW64\taskmgr.exe)
- Download and install windbg.exe. The latest version of this can be found in the Windows 8.1 SDK. Make sure to use the 64 bit version when debugging 64 bit applications.
- Open the dump file in windbg.
- Go to File -> Symbol Path and add the debug symbols of your application. I have added the bin/debug directory as the symbol path.
- Open the command window (if not already open) from View -> Command.
- Type the following commands:
- .loadby sos clr (this loads the sos.dll to help in debugging managed applications)
- .symfix (automatically points to the Microsoft symbol store)
- !dumpheap (this may take some time to run depending on the dump size)
The last command will show you all the objects on the managed heaps, it will also show you the number of objects created for each type and the size of these instances. This is very helpful in narrowing the search radius for the leaky code. The dump looks something like this.
The first column represents the memory address. The second column is the number of instances in memory. The third column is the size these instances are taking up.