eurec4a / eurec4a-intake

Intake catalogue for EUREC4A field campaign datasets
17 stars 19 forks source link

initial botany commit #99

Closed fjansson closed 1 year ago

fjansson commented 2 years ago

Add catalog entry for botany simulations.

observingClouds commented 2 years ago

This is currently failing as it requires a newer version of intake (> 0.6.5) for allowing the reading of uncompressed json. It is already merged but not yet released.

The current workaround is to install intake via pip directly from git:

pip install -e git+https://github.com/intake/intake.git#egg=intake
observingClouds commented 2 years ago

Hi @fjansson, to merge this before intake has been released, could you compress the json file with zip and upload it also to swift? This way we circumvent the current intake bug. Cheers!

fjansson commented 2 years ago

@observingClouds done! The new file is botany_768_params_records.json.zip in the same directory.

fjansson commented 2 years ago

Two more commits: use the zipped json instead of json to get around the intake bug, and added a yaml for the 1536x1536 simulation. So far only the parameter json is there. Then I rebased my branch on top of the new master, hope that went right. Cheers, Fredrik

observingClouds commented 2 years ago

This currently depends on solving #102.

d70-t commented 1 year ago

Ok, now that #102 is fixed, the test is (correctly) failing as there is currently no implementation for testing the availability of json data sources.

observingClouds commented 1 year ago

@fjansson please merge my PR to your branch. It adds a simple test for the json data source. It reads the entire data source into memory which is not perfect, but for the json files that we currently have in the catalog this is totally acceptable.

We might change it to item.discover when we have resolved the last open points in https://github.com/intake/intake/pull/675

fjansson commented 1 year ago

Done. Thanks!

observingClouds commented 1 year ago

Time to work on https://github.com/eurec4a/how_to_eurec4a/pull/81 :smile: