bloomberg / memray

Memray is a memory profiler for Python
https://bloomberg.github.io/memray/
Apache License 2.0
13.36k stars 397 forks source link

Add callback to the Tracker API to post-process the capture file #599

Closed sarahmonod closed 5 months ago

sarahmonod commented 6 months ago

Is there an existing proposal for this?

Is your feature request related to a problem?

It is a fairly common thing for people to want to post-process the capture file, especially when running in a container (for example to send it to a different location over the network for later analysis).

Even though it wouldn't work when a process runs out of memory, it could still be useful to have a way to do this for all other cases of memory tracing.

Describe the solution you'd like

Add a way to provide a callback that would get a path to the capture file so that users can run any arbitrary post-processing on the file, for example sending the file to another location over the network.

Alternatives you considered

No response

micheleAlberto commented 5 months ago

@gusmonod can I pick this up?

micheleAlberto commented 5 months ago

We reflected and wondered if this really makes sense.

We could support a workflow like this where a callback is called passing the output path on exit

output ="my/path/to/capture.bin"
callback = lambda capture_path : do_post_processing(capture_path)
with Tracker(output, post_processing_callback=callback):
    run_my_program()

but this would be fairly equivalent to

output ="my/path/to/capture.bin"
callback = lambda capture_path : do_post_processing(capture_path)
with Tracker(output):
    run_my_program()
callback(output)
sarahmonod commented 5 months ago

Closing as per the above