kalden / spartan

Simulation Parameter Analysis R Toolkit ApplicatioN: Spartan
GNU General Public License v2.0
1 stars 4 forks source link

unexpected decimal values in lhc_generate_lhc_sample_netlogo #12

Closed atfornes closed 6 years ago

atfornes commented 6 years ago

Running lhc_generate_lhc_sample_netlogo with a range and increment in PARAMVALS, the obtained values have unexpected decimals, for instance, the following code

library("spartan")

    FILEPATH <- "./lhc"
    PARAMETERS <- c("param1")
    PARAMVALS <- c("[0,10,2]")
    NUMSAMPLES <- 2
    ALGORITHM <- "normal"
    EXPERIMENT_REPETITIONS <- 1
    RUNMETRICS_EVERYSTEP <- TRUE
    NETLOGO_SETUP_FUNCTION <- "setup"
    NETLOGO_RUN_FUNCTION <- "go"
    MEASURES <- c("count tasks")

 lhc_generate_lhc_sample_netlogo(FILEPATH,PARAMETERS,
    PARAMVALS,NUMSAMPLES,ALGORITHM,
    EXPERIMENT_REPETITIONS,RUNMETRICS_EVERYSTEP,
    NETLOGO_SETUP_FUNCTION,NETLOGO_RUN_FUNCTION,MEASURES)

Results in the following parameters:

param1
0.0122158435406163
0.976000315393321

while only values from {0,2,4,6,8,10} set where expected

kalden commented 6 years ago

Strange. Thanks, will take a look and get back to you

kalden commented 6 years ago

Hi @atfornes, This seems to be an error in the documentation here.

Going through the code, there is no provision for specifying parameter values in this way for latin-hypercube sampling. Using Technique 2, robustness analysis, you can do just that, which is where I think the error must have come from in the docs.

We've always used LHC to generate a number of samples across a range from min and max, and have always left these as double values. There is nothing in spartan at the moment to create samples of integers across the range. So in the vignette PARAMVALS<-c(150,"[10,90]","[10,90]","[5,40]"), first parameter will always be 150, second sampled from 10-90, etc.

If you were wanting to generate netlogo experiment files for this one parameter for values 0,2,4,6,8,10, the methods in Technique 2 will achieve this, rather than Technique 3.

If I haven't quite understood do drop me an email and we can look again.

We'll correct the documentation in the code.

atfornes commented 6 years ago

Thanks @kalden,

Indeed, the confusion came from the documentation for the PARAMVALS definition for lhc_generate_lhc_sample_netlogo function at https://www.rdocumentation.org/packages/spartan/versions/2.3/topics/Technique%205%3A%20SPARTAN%20and%20Netlogo

The experiments I am willing to produce try to study the parameters in a global perspective. The example I gave consist of only one parameter and therefore Technique 2 could be used. However, the model I want to study has more than 20 parameters, and I would like to study the interactions among them. Most of these parameters should have integer values (such as the number of initial turtles in the model).

Is it interesting to try to provide range values for LHC technique? I could help implementing this and would appreciate if you can let me know the relevant sections of the code I have to modify. Otherwise, I will probably implement an external script to convert the double values of the experiments files to integers, which I can later share here.

kalden commented 6 years ago

New functionality written by @atfornes added in version 3.0.1