Episode 7: Intro to vector data

[rogerkuou]

We need to adapt the original Episode 8: Open and Plot Shapefiles in Python.

ToDos:

• Data retrieval via Jupyter inline commands (e.g. via geopands) from open resources:

• Decide the open data to use. Possibledata sources:

  • Crop fields (Basisregistratie Gewaspercelen): https://www.pdok.nl/introductie/-/article/basisregistratie-gewaspercelen-brp-

  • Groud water monitoring well (BRO Grondwatermonitoringnet): https://service.pdok.nl/bzk/brogmwvolledigeset/atom/v2_1/downloads/brogmwvolledigeset.zip

  • (Skip for now) Dikes (BRO Geomorfologische Kaart van Nederland 2019 V1): https://www.pdok.nl/downloads/-/article/basisregistratie-ondergrond-bro-#fa90454e447b478fb2db187bb6fc8a10

• Go through data loading and visualization. Data cropping should probably also be involved for visualizing a huge amount of data. (e.g. the polygons. They are ~0.5gb)
• Crop the polygon data with the bounding box in RD, and save the results for later episodes
  • xmin, xmax = (120000, 135000); ymin, ymax = (485000, 500000)
• Skip the cropping for groundwater monitoring wells since it requires reprojection for cropping

[rogerkuou]

Hi @raar1, glad you are working on this, just some hints on loading and cropping the vector data:

For the crop field polygons, I did something like this:

import geopandas as gpd

# Load all crop field boundaries (brpgewaspercelen)
polygon_cropfield = gpd.read_file("https://service.pdok.nl/rvo/brpgewaspercelen/atom/v1_0/downloads/brpgewaspercelen_definitief_2020.gpkg")

# Define a Boundingbox in RD
xmin, xmax = (120000, 135000)
ymin, ymax = (485000, 500000)
polygon_cropfield_crop = polygon_cropfield.cx[xmin:xmax, ymin:ymax] 

There gpd.read_file takes ~2mins on my local computer. All other operations are quite quick. I think at least for the crop fields data we can load it in this way.

I am still looking into the dike and the groundwater wells.

[raar1]

Thanks very much @rogerkuou ! These are indeed useful pointers

[rogerkuou]

And I also just checked the groundwater monitoring wells, it turned out even the .zip data can be loaded with geopandas:

# Load locations ground water monitoring wells
points_gwm_wells = gpd.read_file('https://service.pdok.nl/bzk/brogmwvolledigeset/atom/v2_1/downloads/brogmwvolledigeset.zip')

I guess if there is only one file within the .zip, geopandas will load it directly. According to the doc , one can use ! to access subfolders under .zip, and use layer= option to access other layers under the .gpkg file. I was having trouble simply because the URL was wrong... I already updated the URL for groundwater monitoring wells.

One thing you also need to pay attention to is that the well points come in another CRS than RD, so you can not direct crop them with the BB I just mentioned. But you can directly load and visualize (though points will overlap each other). Since we won't touch the reprojection in this Episode, I suggest you simply show the data without reprojection/cropping.

And another suggestion is about the dike data. It actually comes in thin polygons but not polylines. I suggest we skip it for now because we may not even want to use it.

I updated the issue description accordingly, to make the action points clearer. Just let me know if you have question/comments/suggestion

[rogerkuou]

Hi @fnattino and @SarahAlidoost, see my comments above for the:

• bounding box
• crop fields polygon loading
• ground monitoring wells loading. (@SarahAlidoost the wells comes in another CRS than RD. So you need to reproject then crop. I would suggest putting its cropping to Ep8 which you are working on.)

[raar1]

@rogerkuou I'm trying to find a replacement for the line elements, since it could be nice to keep the points/lines/polygons coverage of the original episode (not sure if you agree).

I was wondering if we could use something like https://www.pdok.nl/downloads/-/article/nationaal-wegen-bestand-nwb-, specifically the water ways ("Vaarwegen vaarwegvakken (ATOM)" in the Downloads).

The elements are of type LineString, with RD coords so cropping works fine. The plot looks something like:

Is this something we could work with?

[rogerkuou]

Is this something we could work with?

Hi @raar1 , indeed this looks awesome! I think if we can crop it, we can definitely test if this works for the raster cropping episode. And @SarahAlidoost if you have time can you test this?