search

I-GUIDE / CI_Platform

iGUIDE CI Platform Deployment
Apache License 2.0
0 stars 0 forks source link

OSDF integration #15

Open rkalyanapurdue opened 8 months ago

rkalyanapurdue commented 8 months ago
  1. Evaluate utilizing OSDF (https://osg-htc.org/services/osdf.html) to host large public datasets for use in I-GUIDE CI platform.
  2. Plan is to start with WRFHydro outputs from Anvil and move them to OSDF origin deployed on Purdue's CC* storage resource.
  3. Also evaluate and demonstrate Pelican project's fsspec Python library that can help slice NetCDF files similar to kerchunk.
rkalyanapurdue commented 8 months ago

Initial meeting held w/ Brian Bockelman from OSG group and Erik Gough from Purdue to discuss this integration. Next steps:

  1. Wait for StorageClass to be developed for integrating CC* storage with Geddes cloud.
  2. rsync example WRFHydro outputs from Anvil scratch to CC* storage.