Generalize and improve on compress MESA

[mkruckow]

• [x] Reduce the number of filesystem walks from 4 to 2, hence it is a bit faster now
• [x] Generalize the code to be able to be run on any directory level above the MESA runs
• [x] Ensure that only files in the MESA runs are changed without assuming a certain file structure (e.g. not only exclude files in star1, star2 and binary instead only include files in a MESA run file)
• [x] Give more statistical output: number of runs, number of changed files and number of their base directories (the last number is the one shown on the progress bar; the first two are for checks for the user, whether it acts on what is expected)
• [x] Final check, that there are no files to act on after the file actions are completed
• [x] Update documentation
• [x] Added by @astroJeff : catch fails on removals, e.g. because of missing permissions
• [x] Testing, here the new output format from the test:

  remove 0 core dump files in 0 directories of 803 MESA runs 0it [00:00, ?it/s] compress 7194 files in 2026 directories of 803 MESA runs 100%|███████████████████████████████████| 2026/2026 [20:14<00:00, 1.67it/s]

  Compressed MESA tracks Original size 41.3G | Compressed size 10.8G

[sgossage]

A minor QOL thing to add might be to list the full paths of the newly created compressed files, to make it clear to new users where things are stored.

[mkruckow]

A minor QOL thing to add might be to list the full paths of the newly created compressed files, to make it clear to new users where things are stored.

No file gets moved or joint with another file. They are compressed individually and in place, e.g. out.txt becomes out.txt.gz at the same location. Hence, I can add a statement like "in place" and/or "by adding extension .gz". But I think, this might be more something to add to the documentation (which I have done now) instead of the screen output.

[astroJeff]

Update (Apr 25): Waiting on @astroJeff to test.

[astroJeff]

Update (May 9): @astroJeff to run test on Quest

[mkruckow]

Adding a try around the removing is a good idea as an additional improvement to the old code.

[astroJeff]

Running script on Quest in /projects/b1119/POSYDON_GRIDS_v2 right now. Everything looks good and seems to be working well