Tar'ing and copying in run_sim.sh

[pdeperio]

(Simple task for anyone wanting to get more familiar with the workflow.)

Add flag to toggle tar'ing and copying in run_sim.sh here: https://github.com/XENON1T/processing/blob/master/montecarlo/run_sim.sh#L178 such that it can be used locally. Default can be off so as to minimize changes in mc_process.py.

[pdeperio]

@mrcselvi assigned you, but if you see anyone else that can start using the workflow, feel free to delegate

[sthapa]

Just fyi, the current code does tarring in two locations, 178 and 188. In the longer term, when switched over to pegasus we'll need to tar by default since specifying the output files for pegasus is a bit involved and it's easier to just return the results as a single tarball. Not sure if this would remove the need for the flag or not though.

[pdeperio]

ok, ya I had to do this now for EGI since I couldn't copy to Stash and had to retrieve the output locally.

should still have the flag though (for running locally, not on grid).

[mrcselvi]

When I tried to use mc_process.py in the current version on xe-grid01 at LNGS, it didn't work, because of a problem with "Pegasus". @massoli told me that that package is not installed at LNGS ... is there any way to install it, so to have the whole chain also there ?

[pdeperio]

@sthapa is still working to finalize the EGI part of this for next week. can you try the "before_pegasus" branch, which uses just the original glite command?

[mrcselvi]

Yep, the "before pegasus" version works ... it's the one @massoli suggested me to use instead. Thanks !

[pdeperio]

but make sure to use my branch:

git checkout before_pegasus

i added some fixes that Fabio might not be aware of (@massoli, please also check my changes)

[pdeperio]

fixed since integrating EGI and OSG into single Pegasus framework.