informatics-lab / covid19-examples-and-docs

Documentation and examples for the Met Office Informatics Lab COVID-19 response platform and data.
1 stars 3 forks source link

Shapecutter example #10

Closed kaedonkers closed 4 years ago

kaedonkers commented 4 years ago

Add shapecutter example from https://github.com/informatics-lab/covid-19_shapecutter

review-notebook-app[bot] commented 4 years ago

Check out this pull request on  ReviewNB

You'll be able to see Jupyter notebook diff and discuss changes. Powered by ReviewNB.

aaronspring commented 4 years ago

I can provide a much leaner implementation of netcdf aggregation over shapefile regions using https://github.com/mathause/regionmask. This would be without the looping and possibility facilitates the automatisation #9. Shall I start a PR? And if so: where is the data stored? The example here runs with locally downloaded data as far as I can see. @kaedonkers @tam203

tam203 commented 4 years ago

Hi @aaronspring. The data is all available in the blob, README and index will hopefully point you in the right direction. Anything under global_daily would be a good example.

In the above example it is running on our platform that we can add you on to if you'd like. On the platform the data is available on an NFS mounted disk. But due demo purposes you should be able to either down load a few file or even use direct from the blob store. There are some examples of this here

Cheers

aaronspring commented 4 years ago

Thoughts about this implementation? regionmask is well tested.

https://github.com/informatics-lab/covid19-examples-and-docs/pull/16

aaronspring commented 4 years ago

https://github.com/aaronspring/covid19-examples-and-docs/blob/AS_shape_regionmask/examples/1.3.0%20Extracting%20Regional%20Data%20using%20Shapefiles%20alternative.ipynb

tam203 commented 4 years ago

@aaronspring Hi, looks awesome but unfortunaly I'm getting:

'AttributeError: module 'regionmask' has no attribute 'from_geopandas' 

What version are you using? I'm 0.5.0

Screenshot 2020-04-25 at 21 55 24

Cheers.

kaedonkers commented 4 years ago

Hi @aaronspring, thanks so much for this example! This is exactly the kind of xarray based solution I thought would be much simpler but I never got round to 😅

@tam203 It looks like from_geopandas is only a few days old (!) and unreleased. @aaronspring When do you think v0.6.0 of regionmask will be released?

aaronspring commented 4 years ago

Yes. I implement from_geopandas with the help of @mathause. The is no new release of regionmask yet. For now you need to install it from github source

aaronspring commented 4 years ago

Did it work when installing regionmask from source? @kaedonkers @tam203

I will try to get a cloud version running. My laptop and remote internet connection performance are quite poor for this...

mathause commented 4 years ago

You should be able to install regionmask master directly with:

pip install git+https://github.com/mathause/regionmask.git

(Given you already installed all dependenies.) I can make a release if you prefer not to work with a development version, but this may take some days.

tam203 commented 4 years ago

Got a new issue but I've not debugged at all. @kaedonkers

error

If you want to test I've updated the default environment.

tam203 commented 4 years ago

The image didn't work via email so here it is.

error

mathause commented 4 years ago

Ah yes, the error message is missleading (fixed by mathause/regionmask#119). This means that gdf has no column named "name", the part about NaturalEarth can be ignored.

What does display(gdf) and gdf.columns show?

aaronspring commented 4 years ago

I also got this problem. didnt solve it yet. this comes from how the shapefiles are read into geopandas. if only .shp file is present it doesnt work, with some of these other files read_file works but the columns have weird names. locally I just download all these files or often for shapefiles even just a zip file. then it usually works with the expected column names.

working on this tomorrow in https://binder.pangeo.io/v2/gh/aaronspring/covid19-examples-and-docs/AS_pangeo_notebook.