search

HEPData / hepdata_lib

Library for getting your data into HEPData
https://hepdata-lib.readthedocs.io
MIT License
15 stars 39 forks source link

skip overwriting existing output when creating record #179

Closed mhance closed 2 years ago

mhance commented 3 years ago

Hi,

Fantastic package, really made my life significantly easier when generating a pretty big record. Thanks for supporting this!

One thing I noticed when debugging my record is that running convert to create PNG's and thumbnails is (not surprisingly) quite slow. Often the convert commands are overwriting PNG's or thumbnails that have already been created, especially if we're iterating on relatively minor changes in the yaml files.

It would be fantastic to have an option to not overwrite existing PNG files, or to skip PNG file creation altogether. I can see how this isn't something that we'd want by default, but being able to pass in an option to submission.create_files() to skip PNG creation would be super useful for debugging.

Thanks again!

-Mike

AndreasAlbert commented 3 years ago

Thanks for your positive feedback, Mike!

Off the top of my head: We could use a make-like functionality where we only create the PNG if a) it does not already exist or b) if it exists, but it has a modification date that is older than the source PDF. On the other hand, a simple switch might also do it.

kpedro88 commented 2 years ago

Just wanted to second this request.

It could also be useful to provide an option for running multiple imagemagick calls in parallel, which would make even the initial creation of a submission much faster.