Run a first experiment of irrigation planning for a rice crop in Porto-Novo

akouete-kpakpo commented 2 years ago

Tasks

[ ] adapt the code and run python -m nevergrad.benchmark rice_irrigation --num_workers=16 --plot
[ ] analyze output
[ ] make a presentation to the team

Ethel2003 commented 2 years ago

Adaptation of the code and launch of the simulation on rice irrigation

I did the differents modifications and run the code. But the simulation didn't work. It ran for a while but there was an error. This is the error :

Traceback (most recent call last):
  File "C:\Users\EEIA 2021\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\process.py", line 243, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\benchmark\core.py", line 192, in compute
    for local_ind, (index, xp) in enumerate(self):
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\benchmark\core.py", line 148, in <genexpr>
    enumerated_selection = (
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\benchmark\experiments.py", line 1283, in rice_irrigation
    funcs = [Irrigation(i) for i in range(17)]
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\benchmark\experiments.py", line 1283, in <listcomp>
    funcs = [Irrigation(i) for i in range(17)]
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\functions\irrigation\irrigation.py", line 45, in __init__
    self.soil = CABOFileReader(Path(IRRIGATION_DATA_DIR, "soil", "ec3.soil"))
  File "C:\Users\EEIA 2021\nevergrad\venv\lib\site-packages\pcse\fileinput\cabo_reader.py", line 176, in __init__        
    raise PCSEError(msg)
pcse.exceptions.PCSEError: Empty CABO file!
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\EEIA 2021\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main        
    return _run_code(code, main_globals, None,
  File "C:\Users\EEIA 2021\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\benchmark\__main__.py", line 136, in <module>
    repeated_launch(
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\benchmark\__main__.py", line 118, in repeated_launch
    csvpath = launch(
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\benchmark\__main__.py", line 32, in launch
    df = core.compute(
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\benchmark\core.py", line 298, in compute
    dfs = [j.result() for j in jobs]
  File "C:\Users\EEIA 2021\nevergrad\nevergrad\benchmark\core.py", line 298, in <listcomp>
    dfs = [j.result() for j in jobs]
  File "C:\Users\EEIA 2021\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 438, in result  
    return self.__get_result()
  File "C:\Users\EEIA 2021\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 390, in __get_result
    raise self._exception
pcse.exceptions.PCSEError: Empty CABO file!

Analyse of output

Compared to the code output, I think the code gives us a series of characteristics (loss, elapsed_budget, elapsed_time, error, pseudotime, num_objectives, info/sub-optim, seed, symmetry, n_iterations,..... ., optimizer_name) that were evaluated on the different optimizers that were tested on the simulation. These optimizers include: NGOpt10, CMandAS2, CMA, MetaModel, DE. The output of the code should therefore allow us after study to have the best optimizer for the simulation of the irrigation of the crop.

@akouete-kpakpo , @abiolaTresor , @chrichri17 yours views are welcom

chrichri17 commented 2 years ago

Hey @Ethel2003, the useful part of your error is probably pcse.exceptions.PCSEError: Empty CABO file! @akouete-kpakpo is that because we download the ec3.soil for each irrigation ? I mean, the download probably went wrong at some point and an empty file is stored instead ? I can take a closer look 👀

akouete-kpakpo commented 2 years ago

@chrichri17 I agree with you. This is the most likely explanation.

In that case, we should refactor the code to extract a function get_soil_data that won't try to download the soil data if it already exists. Could you handle it: open an issue and make the MR?

akouete-kpakpo commented 2 years ago

@Ethel2003 I've already asked you (cf https://mlforbeninexcellence.slack.com/archives/C03PSF5C01H/p1658403451149279?thread_ts=1658356928.654659&cid=C03PSF5C01H) to push your work.

It is important for us when it comes to debugging. You have to lose the habit of posting the stack of errors only. We'll often want to reproduce your work (maybe the bug is due to some wrong changes in the code on your side and not to the source code itself). In that case, we'll need to switch to your branch. If it is stored on the remote then it will be easy. Else we'll have to make a call with you and look at your changes on your machine. In the latter case, it make us lose time and energy.

So please, push your branch!

akouete-kpakpo commented 2 years ago

To follow up, @abiolaTresor could you check with @Ethel2003 if she had difficulties pushing her branch and opening an MR? Where are you on the task of setting up recurrent meetings to train her on Git / Github (#11)?

akouete-kpakpo commented 2 years ago

Analyse of output Compared to the code output, I think the code gives us a series of characteristics (loss, elapsed_budget, elapsed_time, error, pseudotime, num_objectives, info/sub-optim, seed, symmetry, n_iterations,..... ., optimizer_name) that were evaluated on the different optimizers that were tested on the simulation. These optimizers include: NGOpt10, CMandAS2, CMA, MetaModel, DE. The output of the code should therefore allow us after study to have the best optimizer for the simulation of the irrigation of the crop.

@Ethel2003 thanks for your first analysis. Indeed. But you need to go deeper into the analysis.

have you understood the definition of each characteristic (loss, elapsed_budget, elapsed_time, error, pseudotime, num_objectives, info/sub-optim, seed, symmetry, n_iterations,..... ., optimizer_name) and how it is related to the initial problem / use case?
do you know the algorithm behind at least one of the optimizers you cite: NGOpt10, CMandAS2, CMA, MetaModel, DE. What is the difference between them. Why are you testing all of them for your problem? Maybe it is simpler to start with only one?
What problem these optimizers are competing to solve? Could you formalize it in terms of Optimization (find argmax of func)?

@chrichri17 @abiolaTresor could you help?

chrichri17 commented 2 years ago

In that case, we should refactor the code to extract a function get_soil_data that won't try to download the soil data if it already exists. Could you handle it: open an issue and make the MR?

I'm on it

abiolaTresor commented 2 years ago

To follow up, @abiolaTresor could you check with @Ethel2003 if she had difficulties pushing her branch and opening an MR? Where are you on the task of setting up recurrent meetings to train her on Git / Github (#11)?

Yeah sure, let me know @Ethel2003 if you are having issues with this

ML-for-B-E / nevergrad