Mess with the management of paths

AlexisDrogoul commented 4 years ago

It seems that the management of paths at the simulation level creates some problems... At least experiments should have an idea of the path, even without a simulation created.

As an example, in "Realistic Lockdown Durations.gaml", the dataset is asked before any simulation is created. And this makes the model crash at the very beginning (not to mention that the function, ask_dataset_path, that asks for the dataset is declared in experiment but is using world, which is not available in experiments).

The only way I could make it work was to code, in "Abstract Experiment.gaml", the following:

    /*
     * Gather all the sub-folder of the given dataset_folder
     */
    list<string> gather_dataset_names(string absolute_dataset_fol) {
        write(absolute_dataset_fol);
        list<string> dirs <- folder(absolute_dataset_fol).contents  ;
        write dirs;
        dirs <- dirs where folder_exists(absolute_dataset_fol + "/" + each);
        write dirs;
        return dirs;
    }

    string ask_dataset_path(string relative_dataset_folder_path) {
        string question <- "Choose a dataset:";
        list<string> dirs <- gather_dataset_names(project_path + relative_dataset_folder_path);
        string ds <- user_input(question,choose("", string, dirs[0], dirs))[""];
        return project_path + relative_dataset_folder_path + "/" + ds + "/";
    }

And call it with string shape_path <- self.ask_dataset_path(DEFAULT_DATASET_FOLDER);

In my opinion all these manipulations of the path should be in experiments, not in the model/simulation. And they should only be available to initialize the model/simulation with the correct values. Ideally, then, the management of the dataset path should not be in Parameters.gaml, but in the Abstract Experiment.gaml file...

chapuisk commented 4 years ago

If I understand correctly what you suggest, every experiment (including future ones) should initialize its simulation explicitly with dataset management ?

AlexisDrogoul commented 4 years ago

Hum. Might be cumbersome. You’re right. But then, it can’t be simply managed in simulations. So we need to have some ways to have it in experiments too.

chapuisk commented 4 years ago

I have done both, but in any case, path will not be relatives (as it is the case for import). So, either the simulation is explicitly init from a specific data set using methods in Abstract Experiment.gaml OR you define explicitly the destination toward your use_case

AlexisDrogoul commented 4 years ago

I think I found a way to have the defaults in the experiment and the variables (path and folder) in the simulation. It's a bit weird but seems to work well.

It involves creating, for instance in Constants.gaml, a virtual experiment and defining in this experiment the constants representing the default path and folder. All other experiments will need to inherit from it.

Then, also in Constants.gaml, we redefine the variable called experiment in global and give it the virtual experiment as type.

Once it is done, we can use everywhere experiment.DEFAULT_DATASET_FOLDER or whatever it is called.

chapuisk commented 4 years ago

@AlexisDrogoul I also have a solution (more verbose) with both initialization from Parameters.gaml using DEFAULT and variables (case_study and dataset_folder) that can be overwritten in global section of experiment file AND also providing experiment actions to init within action _init_ of experiment (with default values as arguments) ...... well we are working on the same thing ! You push you're solution ? If not, I'll be testing over all experiments (to be sure that it works for every experiments) and push mine

AlexisDrogoul commented 4 years ago

I wont push mine. Go ahead ! And since you're at it, I'm a bit uncomfortable with the names of all these constants / variables. For example the fact that 'dataset' and 'case study' seem to be synonyms, or that with a name like dataset_folder we don't know if we are relative or absolute (i.e. is it the folder name or the folder path ?) and we don't know if it is the folder containing the dataset (it should end with an "s" in that case) or the folder containing the data of the case study (shapefile, image, etc.).

So, in my opinion, all the variables dealing with the paths should indicate clearly if they are a name (a segment of a path) or a path and, in the case of a path, whether they are project_relative or model_relative...

So, it'd be better to have things like project_relative_path_to_datasets, dataset_name, project_relative_default_path_to_datasets, default_dataset_name ...

chapuisk commented 4 years ago

I add a correction so every experiment now initialize ! However, I still don't get you trick to share relative path to model experiment so dataset path can be relative to model. The same apply for the parameters files in Parameters folder. For example, in Parameter.gaml there is:

string epidemiological_parameters <- DEFAULT_MODEL_TO_PROJECT_PATH+"/Parameters/Epidemiological Parameters.csv";

which only work as long as your experiment is launched from something close to Experiments/my_folder/my_model_with_experiments.gaml

AlexisDrogoul commented 4 years ago

Why don’t you want to use absolute paths ? What is the purpose of having something like DEFAULT_MODEL_TO_PROJECT_PATH, while you can simply access the same file using experiment.project_path ? And using this absolute path would allow the user to use relative paths more easily I guess (with a combination of project path , model path , we can probably check easily if a path exists or not of if it is relative to the model or absolute or not)

AlexisDrogoul commented 4 years ago

Several comments:

this is still not working for "Abstract Experiment with Parameters". (NPE happening)
DEFAULT_DATASET_FOLDER_NAME : why is it interesting to have the name of this folder ? It should better be a path (from the project root) and renamed with DATASETS as it is a path to the folder containing the datasets. So something like DEFAULT_DATASETS_FOLDER_PATH should be more appropriate. Same remark for dataset_folder_project_path. Should be datasets and I'm not quite sure that the sequence of folder / project / path makes any sense for anyone... datasets_folder_path only ? Or datasets_folder_path_in_project ? Or project_relative_datasets_folder_path ?
EXCLUDED_CASE_STUDY_FOLDER_NAME : it is a list, so it must be a plural. EXCLUDED_CASE_STUDIES_FOLDER_NAMES would be better.
DEFAULT_MODEL_TO_PROJECT_PATH: I still really do not see the point of this variable. It would be best to manipulate absolute (in the project) paths, no ? And using experiment.project_path and experiment.model_path, you should be able to transform absolute and relative paths to absolute ones, no ? And, at worst, if you really need such a path, you can easily obtain it by comparing experiment.project_path and experiment.model_path...

chapuisk commented 4 years ago

Overall comment: I have tried to makes thing to ease path management for users (e.g. even with bad names given for case_study_folder_name and datasets_folder_path error or default will be raised/launch) and for us in the future; also trying to have something robust, i.e. that works with default / relevant found in any case. Any way I think that the A to Z documentation to build a new model case study should clarify this !

I do not test "abstract" experiment ... now it works (thanks to the call to experiment.project_path), but talking about name specification, this experiment should not be named Abstract if we can launch it.
Constants are being use for built-in datasets and focus on "names" to ease the thing for users: they just have to mention the name of a case study or drag-and-drop they datasets folder into COMOKIT and tell Gama the name of the folder: e.g. here I can use datasets from the COMOKIT Datasets typing
```
string case_study_folder_name <- "Domiz - refugee camp";
string datasets_folder_path <- "External Datasets";
```
in the global of any experiment (knowing that the folder External Datasets is at the root of the project. If it is not the case, absolute path has to be provided.
Done
I replace reference to this everywhere by experiment.project_path.

Hope this time will be the time to close the issue ...

AlexisDrogoul commented 4 years ago

(SOLVED)

Not yet !

Problem with "Display All Datasets.gaml". NPE almost immediately...

chapuisk commented 4 years ago

@AlexisDrogoul was it on your side, because for me it was ok ? I close the issue...

AlexisDrogoul commented 4 years ago

No. It was because the data files were not generated in the test folder (which will be the case for new users)

Le jeu. 7 mai 2020 à 19:48, Kevin Chapuis notifications@github.com a écrit :

@AlexisDrogoul https://github.com/AlexisDrogoul was it on your side, because for me it was ok ? I close the issue...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/COMOKIT/COMOKIT-Model/issues/23#issuecomment-625234130, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAENNOF6N4KA7ECK3J3IQQTRQKUYJANCNFSM4M2LZ6JA .

-- Representative of IRD in Vietnam and Philippines http://en.vietnam.ird.fr Senior Researcher, Deputy Director, UMI UMMISCO 209, IRD & UPMC, France. http://ummisco.fr Lead architect, GAMA Platform, http://gama-platform.org — alexis.drogoul@ird.fr | http://goo.gl/4ewFdJ [Vietnam]+84944455180 [France]+33608698845 [Address]Appt. 103, Bldg 2G, Van Phuc Diplomatic Compound, 298 Kim Ma, Ba Dinh, Hanoi

COMOKIT / COMOKIT-Model

Mess with the management of paths #23