acocac
commented
2 years ago @jamesdamillington thanks for logging the notebook idea. The outline and suggested datasets look great for me. Some comments as follows:
Outline
Overview of raster data characteristics
what do you mean by raster characteristics? size, number of bands
Reading raster data
Try rioxarray
Plotting categorical raster maps Analysing aggregate change (through bars charts and similar visualisation)
Try Holoviz visualisation toolkits. Most of the existing notebooks in the EnvDS book use them for interactive plotting.
Analysing zonal change (using ancillary vector data)
Use zonal_crosstab from Xarray-Spatial. See here an example in Microsoft's Planetary computer.
Analysing pixel-by-pixel change (including use of sankey diagrams)
Not sure which library is the most optimal, but holoviews seems to support interactive sankey diagrams.
Datasets
All datasets are great and considerations are very relevant. My suggestion is to consider coarser datasets such as MODIS MCD12Q1. The MODIS tiles h17v03 and h18v03 cover the whole UK. The average size of each tile is 6MB per year.
Following the MODIS wildfire notebook which fetches the dataset from NASA’s Earth Data site, find below how I suggest downloading the MODIS land cover dataset for a single year and tile. We'll need to list file names for the remaining years and tiles. You can merge tiles using the merge function in rioxarray (see here).
notebook_folder = './general-exploration-landcover_modis'
if not os.path.exists(notebook_folder):
os.makedirs(notebook_folder)
fnames = ['MCD12Q1.A2017001.h17v03.006.2019196134714.hdf', 'MCD12Q1.A2018001.h17v03.006.2019199221720.hdf']
for fname in fnames:
if not os.path.isfile(os.path.join(notebook_folder, fname)) or os.path.getsize(os.path.join(notebook_folder, fname)) == 0:
username = 'XXX' #replace for your EarthData username if run local or in Binder
password = 'XXX' #replace for your EarthData username if run local or in Binder
fsspec.config.conf['https'] = dict(client_kwargs={'auth': aiohttp.BasicAuth(username, password)})
url = f'https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/6/MCD12Q1/{fname[9:13]}/001/{fname}'
filename = url.split('/')[-1]
with fsspec.open(url) as f:
with Path(os.path.join(notebook_folder, filename)).open('wb') as handle:
data = f.read()
try:
data.decode('utf-8')
raise RuntimeError('Could not download MODIS data! Have you authorized LAADS Web in your Eathdata account above?')
except UnicodeDecodeError:
handle.write(data)
# open a single file
modis_hdf = rioxr.open_rasterio(os.path.join(notebook_folder,fnames[0]))
modis_hdf.LC_Type1.plot()
plt.show()An alternative to fetch datasets is via STAC (see this example). I've explored the catalog of the Planetary computer, but they don't have the MODIS MCD12Q1 product.
Next steps
My suggestion is to take the route of MODIS or another product your feel comfortable with and follow the steps in the submission guidelines section.
As part the submission process, once you use the notebook template in your personal repository, please indicate in the comment box the URL to the repo. Thank you!
ps. apologies for the late reply, but I was participating in some project meetings last week. FYI, I've updated notebook templates and submission guidelines. Contributions to improve them would be welcome. I hope you enjoy the submission process and in general the community-driven aspects of the project (:
jamesdamillington
What is the notebook about?
This notebook introduces raster land cover data with simple manipulation and basic exploratory analysis techniques. The notebook will be based largely on the existing notebooks I have used for teaching and will examine:
There are now many data sources of classified (categorical) land cover data that are useful for Environmental Data Science. These include:
Considerations for deciding which of these sources to use in this notebook include:
Code and packages used in this notebook will initially be those used in the original teaching notebooks, notably:
ndarraysIn time, the code can be changed to use packages from the pangeo ecosystem
Data Science Component
Checklist:
Additional information