ai2cm / fv3config

Manipulate FV3GFS run directories
Apache License 2.0
1 stars 0 forks source link

Separate data storage concerns from code (possibly into a configuration file) #20

Open mcgibbon opened 4 years ago

mcgibbon commented 4 years ago

I think what we may want to do is refactor the internal representation of remote data options to use a user-specific configuration file. That way we as a group can use our own configuration file and handle versioning/updating in any way we want to internally without exposing these possibly rapid changes to the public. The way the code is set up right now is a lot like this (a mapping from option names to remote URLs), so it would not take much work to move this logic into a configuration file.

This was prompted from @ofuhrer's post:

What happens if somebody updates the data or namelists on GCP? Is that "illegal"? Is the data on GCP "versioned" and a release of fv3config always get's the same version of the data (until the user updates fv3config)? How is data provenance handled when using fv3config for a series of simulations?

mcgibbon commented 4 years ago

FV3Config: separate data storage into configuration file