Running pyStemmusScope directly at certain lat, lon: Problems

[prajzwal08]

Hi, I've been using Zampy to download data for a specific location, and it works perfectly on my personal computer. However, I've noticed that downloading data from the CDS API takes a considerable amount of time.

As a workaround, I've discovered that data retrieval from Google Earth Engine (GEE) is significantly faster. The only limitation is that GEE primarily provides data in TIFF format (which I can later translate to NetCDF). However, I've also found that I can export data from GEE in CSV format, which is much faster for my purposes.

Now, I have a couple of questions:

1) I'm interested in making changes to the source code of PyStemmusScope to enable reading data directly from CSV files. Currently, PyStemmusScope requires NetCDF files for forcing data. Could you please guide me on where in the source code I can make these modifications? Or is this good idea or not?

2) I've noticed that there's a version 0.3 of PyStemmusScope and in the folder STEMMUS_SCOPE_Processing-0.3.0\tests, there are several .py files where I can set the latitude and longitude of my location and run the PyStemmusScope model directly. However, I'm not entirely sure how this process works. Could you provide some clarification on how to run the model directly by setting the latitude and longitude? Thank you!

[BSchilperoort]

Hi,

I've been using Zampy to download data for a specific location, and it works perfectly on my personal computer

Nice! That's good to hear.

downloading data from the CDS API takes a considerable amount of time.

Yes this is sadly the case. Although the ADS is the largest bottleneck (for the CAMS CO2 dataset).

I'm interested in making changes to the source code of PyStemmusScope to enable reading data directly from CSV files.

We will be modifying PyStemmusScope to be only compatible to prepared PLUMBER2 formatted netCDF files. I think the best course of action would be to write a script that converts your CSV files to the correctly formatted netCDF file.

If you have a working GEE script & accompanying CSV -> netCDF converter it would be nice if you could share it, either here or by creating your own GitHub repository.

in the folder STEMMUS_SCOPE_Processing-0.3.0\tests, there are several .py files where I can set the latitude and longitude of my location

The test folder only contains code for testing if the PyStemmusScope code works as intended, and is not meant to be used by users :wink:

Could you provide some clarification on how to run the model directly by setting the latitude and longitude?

Currently this is not very easy to do, as with the latest PyStemmusScope release we have not incorporated Zampy yet.

The workflow be like the following once we finish up the missing parts:

1. Download all STEMMUS_SCOPE required data with Zampy.
   • Here you would specify which bounding box you want to use. For worldwide lat/lon locations, you'll have to download the entire global datasets. This is a limitation to how the data providers allow us to download the data. We do not use GEE because it's not open.
2. (with Zampy) convert it to the PLUMBER2 format
3. Point PyStemmusScope to the folder with the PLUMBER2-formatted dataset
4. Select a location in PyStemmusScope.

[SarahAlidoost]

2. I've noticed that there's a version 0.3 of PyStemmusScope and in the folder STEMMUS_SCOPE_Processing-0.3.0\tests, there are several .py files where I can set the latitude and longitude of my location and run the PyStemmusScope model directly. However, I'm not entirely sure how this process works. Could you provide some clarification on how to run the model directly by setting the latitude and longitude?

In addition to Bart's comment, please check the PyStemmusScope documentation on how to run the model on a specific location. You need to set the argument Location in model.setup() to lat/lon values as explained in this issue.

[SarahAlidoost]

Documentation and a notebook is added to pystemmusscope to show how to run the model.