cositools / cosipy

The COSI high-level data analysis tools
Apache License 2.0
3 stars 16 forks source link

Check if the downloaded file are already present in the TS map notebook #162

Closed NicoloParmiggiani closed 3 months ago

NicoloParmiggiani commented 3 months ago

If this notebook https://github.com/cositools/cosipy/blob/main/docs/tutorials/ts_map/Parallel_TS_map_computation_DC2.ipynb is executed multiple times it tries to download the files each time and show errors.

The zipped response file is deleted so the code downloads it again even if the unzipped file is present.

israelmcmc commented 3 months ago

Thanks, @NicoloParmiggiani

I'm working on the SC tutorial and I added this:

from pathlib import Path
import shutil

data_dir = Path("") # Current directory by default. Modify if you want a different path

response_path = data_dir/"SMEXv12.Continuum.HEALPixO3_10bins_log_flat.binnedimaging.imagingresponse.nonsparse_nside8.area.good_chunks_unzip.h5"

# download response file ~839.62 MB
if not response_path.exists():

    response_path_zip = str(response_path) + '.zip'
    fetch_wasabi_file("COSI-SMEX/DC2/Responses/SMEXv12.Continuum.HEALPixO3_10bins_log_flat.binnedimaging.imagingresponse.nonsparse_nside8.area.good_chunks_unzip.h5.zip",response_path_zip)

    # unzip the response file
    shutil.unpack_archive(response_path_zip)

    # delete the zipped response to save space
    os.remove(response_path_zip)

@Yong2Sheng Can you do the same for the tutorial @NicoloParmiggiani mentioned, please?

Yong2Sheng commented 3 months ago

Hi @israelmcmc and @NicoloParmiggiani, Thank you for bring it up! This is a good ideal, I will add this script to the downloading cell.

Yong2Sheng commented 3 months ago

This is fixed in a commit in #150

israelmcmc commented 3 months ago

150 closed this.