check datasets load on azureML

informatics-lab / PangeoLectures

Training material on the principles and practices of The Pangeo Way

2 stars 4 forks source link

check datasets load on azureML #10

Closed stevehadd closed 2 years ago

stevehadd commented 2 years ago

the notebooks were previously run on AWS sagemaker and a local laptop. We should check the code works on azureML too. We may need to create our own azure version of some of the datasets, or use different sources (maybe AI for Earth which is azure based), to do more efficient loading of data.

stevehadd commented 2 years ago

I have checked that notebooks run on AzureML and updated the files in thius commit: https://github.com/informatics-lab/PangeoLectures/commit/cdf86b63ce59ad5c19811367922c50c8d895be46 I have created some issues to address specific things

13 create a proper dask array on azure
14 replace met office intake catalog with era5 data from pangeo catalog. The informatics lab intake catalog no longer works, we should swicth to using the ERA5 data in the pangeo catalog.

informatics-lab / PangeoLectures

check datasets load on azureML #10

13 create a proper dask array on azure

14 replace met office intake catalog with era5 data from pangeo catalog. The informatics lab intake catalog no longer works, we should swicth to using the ERA5 data in the pangeo catalog.