Steven Engelhardt

While reading the paper Dynamic Storage Allocation: A Survey and Critical Review, I quickly focused on the claim that one requires actual memory allocation traces, as opposed to simulated data, in order to properly design an optimized memory allocator. After my experience with Win32 LD_PRELOAD, I knew that I could implement a minimally invasive memory trace mechanism for existing Windows binaries. So I did.

My first task was to write a program which would launch the application we wanted to trace and log its calls to the Windows memory allocation functions. As I recommend on my Win32 LD_PRELOAD page, I used Microsoft Research’s Detours, rather than Win32 LD_PRELOAD, for the mechanism to intercept Windows function calls. The only functions I chose to intercept were HeapAlloc and HeapFree; these functions seem to be the workhorses of the Windows memory allocation world.

When my code detects a call to HeapAlloc or HeapFree, it logs basic information about the call to a file and then forwards the call to their real implementations. Reentrancy was an issue; the process of logging could potentially cause another allocation, so I had to guard against that. A more robust application would also consider multithreading scenarios more carefully.

For performance and space considerations, I decided the allocation logger would write log entries as binary records into a file. An allocation log record is 17 bytes long; a free, 13 bytes. However, memory operations are so frequent that a one minute browsing session in Firefox (including a quick visit to Google Reader), generated a 14 megabyte log file. Furthermore, the logger introduced a noticeable, although not drastic, performance hit.

With the Firefox log file in hand, I wrote a few post-processing scripts in Python to calculate some useful data, including total cumulative memory usage and a memory allocation frequency histogram. I then used Gnuplot to graph the results.

Here is the total cumulative memory usage graph for my short Firefox browsing session:

Here is the memory allocation frequency histogram. Note the log-log scale; Firefox (like most applications) is so dominated by very small allocations that the graph is useless without it. The graph has a huge spike of allocations with very small object sizes and, somewhat interestingly, a moderate one with sizes just under 10 KB.

Here is the source code to the memory trace logger.