search

gulfofmaine / sdm_workflow

A repository to help streamline the species distribution model development and prediction workflow.
MIT License
2 stars 0 forks source link

CMIP6 Data Download #22

Closed dzaugis closed 3 years ago

dzaugis commented 3 years ago

I put a .xlsx file in the CMIP6 box folder (CMIP6_GCM_params.xlsx) that describes how to download the CMIP6 data from the WRCP portal and includes a page with the GCM parameters for the models we pull. The Downloaded page should be kept updated with the models we have downloaded and are in the process of downloading. Can someone check if it has been uploaded to box and if you can edit it from box?

Here is the link

aallyn commented 3 years ago

Awesome! When I click the link, a new browser window opens for me to log into Box Online, but then can only edit the file if I click Open in Google Sheets. Not a problem to go that route I don't think... anyone else?

dzaugis commented 3 years ago

I found a couple bugs in Regrid_script.py. I am working them out, and it only slightly impacts the downloading process.

One bug is related to writing large files. I am getting a memory error. I am using dask and opening files in chucks, which I think should let me write files that are too large for memory, but I am still having issues.

The second is related to regridding. When using cdo remapdis (inverse distance weighted method, models that have curvilinear grid are regridding well and the output grid is as expected. Models that have a different (non 1x1) rectangular gird become larger and encroach over the land. Using bilinear interpolation, the encroachment over land does not appear.

dzaugis commented 3 years ago

We decided to access the CMIP6 models through google. The Pangeo project has worked with google to host the models as zarr stores. The models download script is ZarrCMIP6.py (still a work in progress). The bounding box of the download is 20-70 degN and -120- -50 degW.