fatiando / rockhound

NOTICE: This library is no longer being developed. Use Ensaio instead (https://www.fatiando.org/ensaio). -- Download geophysical models/datasets and load them in Python
BSD 3-Clause "New" or "Revised" License
34 stars 15 forks source link

Add LITHO1 model to Rockhound #73

Closed aguspesce closed 2 years ago

aguspesce commented 4 years ago

Add function to fetch and load data from LITHO1 model: an updated crust and lithospheric model of the Earth (https://igppweb.ucsd.edu/~gabi/litho1.0.html). The LITHO1.0 model is a 1° tessellated model of the crust and uppermost mantle of the earth, extending into the upper mantle to include the lithospheric lid and underlying asthenosphere. The data for each layer is fetched and loaded into a xarray.Dataset.

Fixes #70

Reminders

aguspesce commented 4 years ago

Hello @leouieda and @santisoler,

I have made the fetch function for the LITHO model, but I think that it has a problem.

LITHO model has around 40900 node files, each file contains the information of all the boundaries for that node. So, the function downloads the model very fast, but it takes a long time to read all the node files using pandas.read_csv. It takes about 20 minutes to return a xr.Dataset.

What do you think?

leouieda commented 4 years ago

Ouch, this is a tricky situation. What if we sent them a netCDF file (made from the xarray.Dataset) along with a Python script to generate it? The authors might be interested in putting it up on their website and using the script to update it in the future.

aguspesce commented 4 years ago

Hello @leouieda With the latest commit made by @santisoler, the model loads in a few minutes, but it would be better if the model was already in netCDF as you suggested. Anyway, we should write to Gabi Laske asking her to add a CC license, and we can comment her about the xarray.Dataset.

santisoler commented 4 years ago

As @aguspesce correctly said, I refactored the function so the node files are loaded through the standard open() function, and the values are parsed from each line. That speeds up the loading process, so the xr.Dataset can be built after 2-3 minutes. This situation is not ideal and would be way better to have a netCDF file to directly download.

While we wait for Gabi Laske's response, maybe we could still finish this PR and merge it. So anyone who wants to work with LITHO1.0 can do it. In fact, finishing the PR (adding tests functions and improving docs) will ensure we are loading the data correctly.

What do you think? @aguspesce @leouieda

aguspesce commented 4 years ago

I think that is a good idea. What do you think @leouieda ? I'll try to do the test and the docstring in this week