search

DrylandEcology / rSFSW2

rSFSW2: A R package to create soil water balance simulation experiment
GNU General Public License v3.0
8 stars 7 forks source link

`recreate_dbWork`: may be very slow and wasting time #312

Open dschlaep opened 6 years ago

dschlaep commented 6 years ago

Problem: If opt_behave[["keep_dbWork_updated"]] is false, then recreate_dbWork is called each time before simulation runs are made. If opt_out_fix[["use_granular_control"]] is true and there is a lot of output already present in dbOut of a large project, then to "update granular-level information for output table[s]" takes a lot of time, e.g., 7 minutes per table for one large project with about 8.7 million Pids. recreate_dbWork recreates dbWork whether it is up-to-date.

--> Do something so that dbWork knows whether or not it is up-to-date and only recreate if out of date.

dschlaep commented 6 years ago

Commit https://github.com/DrylandEcology/rSFSW2/commit/535af108efd3f6b20623921f5601c7a0c6711704 didn't satisfactorily address this issue -- keep open #312

dschlaep commented 6 years ago

opt_behave[["keep_dbWork_updated"]] is true doesn't work (well) in parallel runs --> set it to false for now until this is addressed