eurec4a / eurec4a-intake

Intake catalogue for EUREC4A field campaign datasets
17 stars 19 forks source link

Add cloudradar of RV-Meteor #45

Closed jroettenbacher closed 3 years ago

jroettenbacher commented 3 years ago

I updated the catalog to include the RV-Meteor. Added a folder for the whole ship and one catalog file for the cloudradar. More instruments from the ship can be added in this folder.

jroettenbacher commented 3 years ago

The test probably fails because the path I updated will be the new path to the files, once Vincent reorganized the folders in Aeris.

jroettenbacher commented 3 years ago

Thank you very much for the contribution 🎉

I've attached two specific comments below. The issue with the failing test currently has been an update of the intake library which required to update our requirements.txt file. I've done that and merged the current master branch into yours.

The new test is currently running, but I'd expect it to fail now due to the not yet updated file paths on Aeris. In order to handle this, there are a few options:

* We can wait to merge the PR until the paths have been updated

* You can change the paths to their current locations and file a new PR once the paths are updated

* It looks like the `high_res` files are already in place, so you could also reduce this PR to `high_res` files only and create a new one once the `low_res` files are in place as well.

Concerning the filepaths I would wait until the new ones are in place as it is the easier solution

jroettenbacher commented 3 years ago

I would like to leave some example code for reading the radar data, as each file has around 500 MB which for a remote source will lead to long wait times if one doesn't use dask. @d70-t How would I do that? Add a README.md to the Meteor folder?

d70-t commented 3 years ago

I would like to leave some example code for reading the radar data, as each file has around 500 MB which for a remote source will lead to long wait times if one doesn't use dask.

We started to build the How to EUREC4A which should evolve into a lager executable book containing examples of how to interact best with the EUREC4A data. I think your example code would fit into that perfectly.

As that thing is now growing, we'll probably move it to a little bit more permanent location soon.

A README in the folder will most likely go unnoticed, because typical users of the intake catalog will (hopefully) never have to look into these folders.

d70-t commented 3 years ago

I like that you introduced the version parameter. This is probably a very elegant solution for other datasets as well. We'll just have to update the default parameter, once a new version is published.

jroettenbacher commented 3 years ago

I would like to leave some example code for reading the radar data, as each file has around 500 MB which for a remote source will lead to long wait times if one doesn't use dask.

We started to build the How to EUREC4A which should evolve into a lager executable book containing examples of how to interact best with the EUREC4A data. I think your example code would fit into that perfectly.

As that thing is now growing, we'll probably move it to a little bit more permanent location soon.

A README in the folder will most likely go unnoticed, because typical users of the intake catalog will (hopefully) never have to look into these folders.

I don't quite get the How to EUREC4A thing running. I guess it would be easier, if I just share my example with one of the maintainers and you can then include it where you deem it to fit?

d70-t commented 3 years ago

Thanks for the update. And also for reverting the README-file. I think this is really not the right place for it to go.

@tmieslinger is currently in the process of reorganizing the entry into the How to EUREC4A at tmieslinger/how_to_eurec4a#25. Once that is in shape, we also plan to move the repository to the eurec4a organization on github and hopefully an entry into the book-business will be easier.

Anyways, once the Aeris service is up and running again, I can also assist you in porting your example into the book.

Regarding this PR, I think this is mostly complete :tada:, we just need to wait for the server to come online again and the files to be in place so that we can re-run the tests and merge it once this succeeds.