python memory leak debugging

ah, yes, the wonders of “automatic memory management”… got off the horn with a good mate and a former colleague who’s struggling with a ugly memory leak in a bit of python code he needs to hammer into shape.

cobbled together the following pointers for him. and it’s quite a shame that there’s a bunch of them too, because none of them are really all that good…

• tracemalloc:
  also merged upstream into python std lib as of python v3.4 (docs). gives tracebacks to where objs were allocaled, and per-source-line allocation stats dumps. and diffs between two heap states, of course. it even has a QT GUI for… er… wimps

• memory_profiler:
  apparently a subset of the capabilities of tracemalloc (e.g. per-line stats), simpler, finer-grained selection of what to trace, some graphing capabilities.

• objgraph:
  grand-daddy of object reference graphing tools, allows you to see which objects are pinning which other objects, graphically (you’ll need graphviz). full memory leak debugging “tutorials” using objgraph that i started off with back in the day were here and here (well, yes, a bit dated, but i’m afraid leak hunting hasn’t really changed much since then)

• muppy:
  slightly different take on the same, with diffing between process state/heap snapshots, etc., reference browser and optional simple GUI.

they may not be as dead simple to use as valgrind to spot mem leaks in C¹, but between tracemalloc and objgraph you can usually nail the problem reasonably quickly.