search

NCAS-CMS / cf-view

Exploration and plotting GUI for netCDF and Met Office format data
https://ncas-cms.github.io/cf-view/build/
MIT License
3 stars 4 forks source link

configuration option for coastline data in cf-view setup #21

Open bnlawrence opened 2 months ago

bnlawrence commented 2 months ago

When cartopy first draws a coastline, it looks for coast line data and downloads it. For low resolution data, that's ok, but it can be time consuming for high resolution data.

It might be that there is a central copy of such data in some sites, and it might be that one is using cartopy from within a container, as we do with the cf-view datatools container. In such cases it would be helpful to have a configuration option for cf-view to declare where the data is (or is wanted to be) so as to avoid repeated and/or unnecessary downloads.

At the moment, things seem to end up in the users ./local/share/cartopy directory, but if that's a docker container, then it's clearly not persistent. There is some detail here as to how to do things differently.

sadielbartholomew commented 2 months ago

Thanks @bnlawrence. I have noticed this issue too (being spammed by the message telling me a feature is being downloaded in cf-plot, at least).

I think this is more of a cf-plot issue, or at least a joint consideration, since the downloading happens directly (e.g. using interactive Python) with cf-plot, too, and isn't related to the UI aspects, so if you don't object I will move the Issue over to the cf-plot repo.

There is some code I have seen in the cf-plot script which I think is intended to try to cache any downloads, but I have my doubts as to whether it works given my recent experience with running scripts downloading Cartopy 'features' e.g. country borders and coastlines etc.:

https://github.com/NCAS-CMS/cf-plot/blob/e64a745f8a6cdf532280430a96cc66f229854087/cfplot/cfplot.py#L65-L72

where the pre_existing_data_dir is designed to store that data (see the documented key here), and where data_dir is the fallback according to those docs.

It might be that there is a central copy of such data in some sites, and it might be that one is using cartopy from within a container, as we do with the cf-view datatools container. In such cases it would be helpful to have a configuration option for cf-view to declare where the data is (or is wanted to be) so as to avoid repeated and/or unnecessary downloads.

I will investigate whether that code is doing anything and then we can think about how best to manage keeping downloaded data in a store generally, with support that will work for containers and as a site/group-wide approach and not just on an individual user basis, with cf-view in mind.