search

SWIFTSIM / pipeline

Pipeline for rapidly analysing SWIFT simulation outputs, using swiftsimio and velociraptor-python
Other
1 stars 4 forks source link

Fast mode #30

Closed bwvdnbro closed 2 years ago

bwvdnbro commented 2 years ago

This PR implements a fast mode for swift-pipeline. In this mode, no scripts are run. This is useful for very large runs, where running the scripts would be too expensive.

Images that are generated by scripts are not included on the web page when running in fast mode. The web page title displays "FAST mode" to make it clear that this is how the pipeline was run.

I also sneaked in a formatting script, because we did not have one yet.

Moyoxkit commented 2 years ago

One thing that would be useful for running the pipeline for calibration, is to also have a toggle in the fast mode that stops the pipeline from making plots at all. I recently found that nearly half of my cosma6 file-number quota was filled up with pipelineplots.

bwvdnbro commented 2 years ago

Looking at the code, it will be very hard to disable making the plots without also disabling the .yml output, but it should be relatively straightforward to not save the figures. That would still cause a lot of overhead though.

bwvdnbro commented 2 years ago

I have added the --no-plots option. This requires a change in velociraptor-python, which is currently still in a separate branch, no-plots.

Moyoxkit commented 2 years ago

Great, thanks!