feat: if files exist in ostore and not locally will pull from ostore …

bcgov / nr-rfc-climate-obs

Transition of the existiing climate observations data pipeline to enable running off prem

Apache License 2.0

1 stars 0 forks source link

feat: if files exist in ostore and not locally will pull from ostore … #20

Closed franTarkenton closed 1 year ago

franTarkenton commented 1 year ago

…vs upstream remote (hpfx)

franTarkenton commented 1 year ago

This PR includes changes that will cause the data pull to pull from object store if the data exists there vs remote. If data does not exist in ostore it will continue to do a pull push from hpfx to ostore. ostore pull should be faster than an hpfx pull