HakaiInstitute / hakai-erddap

Hakai Datasets that are going into https://catalogue.hakai.org/erddap/
1 stars 0 forks source link

Maintain Datasets Files between the different servers #173

Closed JessyBarrette closed 7 months ago

JessyBarrette commented 8 months ago

ERDDAP is generally handling data from

For the files we have as of now a copy on each server:

Typically we may not always want the development branch to present the same data as production so theres occasions where we would point an dataset to different files in production vs development.

So ideally it would be great to have the /data/erddap_data folder been a mounted volume on each server and potentially have a second development volume for just development purposes. This last one could be potentially on the actual server if we want.

JessyBarrette commented 8 months ago

Looking further into this, it seems like both ERDDAP data files on the respective servers are coming from an external mounted drive already:

goose:

/dev/nvme1n1                                              131919812  91457080         34742196  73% /data2
/dev/nvme4n1                                              514937088 245891732        242814572  51% /algae_explorer

catalogue

/dev/nvme1n1                                               65531436  48159560         14000052  78% /data
fs-0da5688cab7d32c08.efs.us-east-1.amazonaws.com:/ 9007199254739968 815926272 9007198438813696   1% /mnt/efs/algex
JessyBarrette commented 7 months ago

This issue is now resolved both files from goose and hecate were both copied to EFS mounts both mounted on the new erddap server