search

mg5amcnlo / mg5amcnlo

Other
58 stars 32 forks source link

Parallelize untar operation in case of gridpack #107

Closed Qubitol closed 3 days ago

Qubitol commented 2 weeks ago

This PR improves the untarring of the zip files created when using the gridpack option, by parallelizing the operation using xargs.

On a separate, but related, issue, the unzipping is performed by following lines of bin/internal/restore_data:

gunzip \$j
cp $1_ftn26 ftn26
gzip $1_ftn26

However, re-zipping the file can be avoided by using the --keep option of gunzip, maybe we can save even more time.

oliviermattelaer commented 4 days ago

Hi Daniele,

Yes you can use the --keep this is a very good idea!.

Then this is good to merge.

Thanks a lot,

Olivier

Qubitol commented 3 days ago

Hi Olivier,

I added the --keep and changed the copy to an mv in the line next to it. If you agree, we can merge, but apparently I don't have write access to the repo.

Thanks.

Daniele

oliviermattelaer commented 3 days ago

Thanks, I have done it myself then :-)