We are currently archiving full model objects. They are large. This slows setup substantially, won't be sustainable, and aren't necessary for most folks. Start archiving just the key pieces that are useful for most applications (probably following EFI standards).
Options
Only archive *_model_fits.json objects locally on HiPerGator + backup
Archive only to the forecasts repo/Zenodo where they aren't currently downloaded
Option (2) seems feasible for now and keeps archiving full objects.
Checklist for Option 2
[x] Modify archive_hipergator.sh to copy files from new fits directory to forecasts/portal/ along with those from the casts directory
[x] Modify portal_weekly_forecast.sh to not delete the portalPredictions directory (where the fits directory will be located). Instead fetch and the reset --hard upstream/master. We could pull but risks a history change breaking things (see note below).
[x] Add fits directory to .gitignore
[x] Move current *_model_fits.json objects from casts to fits on HiPerGator
[x] Remove *_model_fits.json objects from casts in portalPredictions
[x] Move any new *_model_fits.json objects added by weekly forecasts before portalcasting update
[ ] Remove *_model_fits.json from repo history (Blocked by https://github.com/weecology/portalcasting/pull/179 since news _model_fits.json objects will current still be added. Only want to rewrite the history once. Need to make sure this doesn't break the update to portal_weekly_forecast.sh since a basic pull won't work with history rewrite). This isn't essential since the archive doesn't care about the history, but it will make the repo itself much more light weight.
We are currently archiving full model objects. They are large. This slows setup substantially, won't be sustainable, and aren't necessary for most folks. Start archiving just the key pieces that are useful for most applications (probably following EFI standards).
Options
*_model_fits.json
objects locally on HiPerGator + backupforecasts
repo/Zenodo where they aren't currently downloadedOption (2) seems feasible for now and keeps archiving full objects.
Checklist for Option 2
archive_hipergator.sh
to copy files from newfits
directory toforecasts/portal/
along with those from thecasts
directoryportal_weekly_forecast.sh
to not delete theportalPredictions
directory (where thefits
directory will be located). Insteadfetch
and thereset --hard upstream/master
. We couldpull
but risks a history change breaking things (see note below).fits
directory to.gitignore
*_model_fits.json
objects fromcasts
tofits
on HiPerGator*_model_fits.json
objects fromcasts
inportalPredictions
portalcasting
releaseportalcasting
Docker container*_model_fits.json
objects added by weekly forecasts beforeportalcasting
update*_model_fits.json
from repo history (Blocked by https://github.com/weecology/portalcasting/pull/179 since news_model_fits.json
objects will current still be added. Only want to rewrite the history once. Need to make sure this doesn't break the update toportal_weekly_forecast.sh
since a basicpull
won't work with history rewrite). This isn't essential since the archive doesn't care about the history, but it will make the repo itself much more light weight.