Closed abigailsnyder closed 4 years ago
only test failing is the devel on macOS, due to ncdf4
as usual.
Once PR is merged in,
@abigailsnyder will use the new function to create reduced size versions of all emulators in /pic/projects/GCAM/GE/drought-expt/fldgen-emulators/
with the correct variable names. Not updating any of the trained emulators contents, not changing science. Just saving a copy of what is already being used with fewer things in it.
an intermediate fix to https://github.com/JGCRI/fldgen/issues/25
Reducing what we save from a trained emulator to the bare bones list entries needed for generating new fields. This takes the ISIMIP GFDL trained emulator from 5.6gb for everything to 2.1gb. Hopefully this is small enough to work in the Cassandra pipeline.
The scripts in
fldgen/inst/scripts
are copies of the same files in/pic/projects/GCAM/GE/drought-expt/fldgen-emulators
, withfldgen/inst/scripts/train-emulators.R
updated to include a call to the newemulator_reducer
function and save the smaller emulators. So in theory those can be re-built from scratch, or the existing emulators in/pic/projects/GCAM/GE/drought-expt/fldgen-emulators
can each be loaded, reduced, saved with a different script, following what has been added tofldgen/inst/scripts/train-emulators.R
.Note that when you load an RDS object from
fldgen/inst/scripts/train-emulators.R
, it will come in with the namereducedEmulator
and notemu
. So either the python code has to be adjusted or this piece of code in the training script:Would have to be redone as
Left distinct for now, so clear in the fldgen package and cassandra users can amend according to their own preference.
The pointers in the
cassandra
directory on pic will have to be updated to point to these smaller emulators.@crvernon will probably fail on the development version of R for mac tests, since the
ncdf4
package hasn't been updated for that yet. We like to run that test and check the failure to keep an eye on potential issues coming down the road.passes local package check at least. Fingers crossed behaves here.
also removes functions that are never used that caused issue https://github.com/JGCRI/fldgen/issues/40