add remote url datasets to master catalog

aaronspring commented 4 years ago

adds atm: HadCRUT4
ocean: HadISST1, GISTEMP, HadSST4
marine biogeochemistry: SOM_FFN, CSIR-ML6

Closes: #113

open questions:

how much predefined plots? I would say optimal but recommended
how to test?
add data source type tag in metadata? @jhamman which could be used to test dataset availability with xr.open_dataset(url.strip('simplecache::'), chunks={}) without downloading, now with caching to_dask() is not lazy anymore
should caching options be required? my opinion not, but really nice to have, but laziness of to_dask breaks
how many options to set in catalog? I set use_cftime=True to make hvplot work but hvplot seems to be buggy with the new matplotlib version
which tags to set?

aaronspring commented 4 years ago

having caching in the urls leads to issues on the pangeo cloud when using a cluster: https://github.com/intake/intake-xarray/issues/86

rabernat commented 4 years ago

Hi @aaronspring - sorry this PR was not responded to for so long. I'd love to help move this idea forward however I can.

aaronspring commented 4 years ago

thanks Ryan, I first want to resolve intake/intake-xarray#86 and https://github.com/holoviz/hvplot/issues/491. once my knowledge and usage of hvplot and intake with caching is more stable, I will re-open

pangeo-data / pangeo-datastore

add remote url datasets to master catalog #114