hzambran / hydroPSO

Model-Independent Particle Swarm Optimisation for Environmental Models
https://cran.r-project.org/package=hydroPSO
GNU General Public License v2.0
36 stars 18 forks source link

Plotting ggof when fn!="hydromod" #8

Open jthurner opened 7 years ago

jthurner commented 7 years ago

Calling plot_results does not produce the sim-vs-obs ggof graph when fn!=hydromod because read_results can not obtain the best model output time series.

When fn=hydromod, fn yields a list containing 1) the simulated values and 2) the GoF value. For each particle, the simulated values are assigned and then written into Model_Out.txt. When calling plot_result, this file is then read in by read_out which selects the best model output time series by min/max GoF.

If fn!=hydromod, fn should "return a scalar result" which represents "both model output and its corresponding goodness-of-fit measure". In this case model output is set to GoF, and read_out can not obtain the best model output because it isn't saved for each particle.

Some ideas for allowing plot_results to work with output created by fn!=hydromod:

  1. Read best-sim from file in read_out Analog to obs, read the file BestModel_out.txt (which is also created if fn=hydromod) in case Model_out.txt contains only GoF. This would also provide an easier/faster alternative to obtaining best-sim from ModelOut.txt.

  2. Allow fn to return both sim and gof Analog to hydromod, save model output per particles. Would enable read_out to look up the best model output.

  3. Add sim (and obs) as optional arguments to plot_results Simple and quick fix, as in 774d01db0f34d10dcb31cb4d6ca857b89c5a527e. It would then be up to the user to directly supply sim and obs.

1) and 2) would still require the user to call another custom function which runs the model with the best paramters returned from hydroPSO and writes out the best-sim and obs text files before being able to call plot_results. Doing this from hydroPSO directly is currently not possible because unlike with fn=hydromod, the obs parameter name is unknown (and fn currently does not return sim). I suppose it would be possible to pull obs from the ellipsis if it exists or create a stripped-down version of hydromod for running R code instead of external commands.

In general, I think it would be great to make saving model output per particle optional and implement 1) as an alternative way to supply the best model out. One would lose the ecdf plots, but improve performance by not writing the full sim to disk each model run and having to process the resulting ModelOut.txt which can easily grow to several gigabytes. This would possibly involve setting model.out to NA or similar instead of copying GoF, because at the moment plot_results exits with error once it hits the ecdf plots if model.values contains GoF instead of the full sim time series.

jthurner commented 6 years ago

See PR #9 for an implementation of idea 2.

I think this is a good fix regardless of other possible improvements.