@humnaawan I would like to suggest that the csv ditherfiles you produce be gzip compressed so that they are easier to store and transfer around. This will also be useful when we put them up as dataproducts at zenodo following our current trends.
There are two methods of doing this, Add something to the script saving the files
like for file in `ls descDither*.csv`; do echo "gzipping $file"; gzip $file; done
(a little inelegant but faster)
or adding a compression option to the saving (simpler, but takes a bit more time).
I have been doing this for my own purposes, so can add in these options, you can choose.
@humnaawan I would like to suggest that the csv ditherfiles you produce be gzip compressed so that they are easier to store and transfer around. This will also be useful when we put them up as dataproducts at zenodo following our current trends.
There are two methods of doing this, Add something to the script saving the files like
for file in `ls descDither*.csv`; do echo "gzipping $file"; gzip $file; done(a little inelegant but faster) or adding a compression option to the saving (simpler, but takes a bit more time).I have been doing this for my own purposes, so can add in these options, you can choose.