ai2cm / fv3net

explore the FV3 data for parameterization
MIT License
16 stars 3 forks source link

Clean up I/O routines #88

Open nbren12 opened 4 years ago

nbren12 commented 4 years ago

Currently, I/O is strewn around this code, base and we keep reinventing the wheel as a result. We are using many different ways of interacting with the filesystem and GCS. These include

Obviously this situation is much more complicated than the functionality we need. I think we should develop a coherent API for I/O operations. Some of these are specific to GFDL data (e.g. tiles/subtiles) others are more general purposes (e.g. opening a remote netCDF file lazily). I think intake is designed to solve this problem, but we can always write our own custom solutions.

nbren12 commented 3 years ago

I think vcm.convenience.get_root should be deleted.

nbren12 commented 2 years ago

Still a bit of an issue imo.