GregorCH / ipet

Interactive Performance Evaluation Tools for Optimization Software
MIT License
26 stars 6 forks source link

make ipet-parse script multithreaded and add a tqdm progress bar #92

Closed GregorCH closed 3 years ago

GregorCH commented 3 years ago

what the description says. I parallelized this script for two reasons:

  1. It is much faster (n times faster than before)
  2. It consumes less memory than the previous variant, because an experiment and the contained TestRun object is freed immediately after it was parsed and stored to disk, instead of continuing to parse until all files were processed.

Drawback: Files such as meta data, error files etc. are simply discarded if passed to the script. However, I believe this is the same behavior as before. In order to associate a test run with an OUT file and completely differently named ERR/META/whatever file, one would need to setup a custom TestRun object writing custom Python code.

Check out the progress bar.

fschloesser commented 3 years ago

What happened to travis?

GregorCH commented 3 years ago

I dont know. Two weeks ago, everything was fine.