ploomber / ploomber-engine

A toolbox 🧰 for Jupyter notebooks 📙: testing, experiment tracking, debugging, profiling, and more!
https://engine.ploomber.io
BSD 3-Clause "New" or "Revised" License
59 stars 14 forks source link

Modify --profile-runtime and --profile-memory plot file location #81

Closed avivajpeyi closed 10 months ago

avivajpeyi commented 11 months ago

Similar to #78, it would be nice if users can also specify a different location for the plots (or even just an option to disable the plots, while still saving the profiling data).

This can be helpful to keep all plots/profiling data in separate locations from the notebook-outdir (to prevent cluttering the notebook outdir if there are lots of parameterized notebooks present)

Current engine:

toi_0-memory-usage.png
toi_0-runtime.png
toi_0-profiling-data.csv
toi_0.ipynb
toi_1-memory-usage.png
toi_1-runtime.png
toi_1-profiling-data.csv
toi_1.ipynb
toi_2-memory-usage.png
toi_2-runtime.png
toi_2-profiling-data.csv
toi_2.ipynb
...

If the plots (and profiling data) can be saved elsewhere:

toi_0.ipynb
toi_1.ipynb
toi_2.ipynb
...

I think it would be easy to do either of the above, and I would be happy to implement this.

@edublancas, which of the two do you think would be better?

1) Add a --disable-profiling-plots arg 2) Modify --profile-runtime and --profile-memory to accept a path (similar to #78)

edublancas commented 11 months ago

Modify --profile-runtime and --profile-memory to accept a path (similar to Modify --save-profiling-data to accept File Path(Ploomber Engine CLI) #78)

I like this idea, feel free to open a PR!