Closed aguspesce closed 2 years ago
Hello @leouieda and @santisoler,
I have made the fetch function for the LITHO model, but I think that it has a problem.
LITHO model has around 40900 node files, each file contains the information of all the boundaries for that node. So, the function downloads the model very fast, but it takes a long time to read all the node files using pandas.read_csv. It takes about 20 minutes to return a xr.Dataset.
What do you think?
Ouch, this is a tricky situation. What if we sent them a netCDF file (made from the xarray.Dataset) along with a Python script to generate it? The authors might be interested in putting it up on their website and using the script to update it in the future.
Hello @leouieda
With the latest commit made by @santisoler, the model loads in a few minutes, but it would be better if the model was already in netCDF as you suggested.
Anyway, we should write to Gabi Laske asking her to add a CC license, and we can comment her about the xarray.Dataset
.
As @aguspesce correctly said, I refactored the function so the node files are loaded through the standard open()
function, and the values are parsed from each line. That speeds up the loading process, so the xr.Dataset
can be built after 2-3 minutes. This situation is not ideal and would be way better to have a netCDF file to directly download.
While we wait for Gabi Laske's response, maybe we could still finish this PR and merge it. So anyone who wants to work with LITHO1.0 can do it. In fact, finishing the PR (adding tests functions and improving docs) will ensure we are loading the data correctly.
What do you think? @aguspesce @leouieda
I think that is a good idea. What do you think @leouieda ? I'll try to do the test and the docstring in this week
Add function to fetch and load data from LITHO1 model: an updated crust and lithospheric model of the Earth (https://igppweb.ucsd.edu/~gabi/litho1.0.html). The LITHO1.0 model is a 1° tessellated model of the crust and uppermost mantle of the earth, extending into the upper mantle to include the lithospheric lid and underlying asthenosphere. The data for each layer is fetched and loaded into a xarray.Dataset.
Fixes #70
Reminders
make format
andmake check
to make sure the code follows the style guide.doc/api/index.rst
.