search

ec-jrc / lisflood-code

Lisflood OS - LISFLOOD
https://ec-jrc.github.io/lisflood
European Union Public License 1.2
119 stars 47 forks source link

Question on Transient Land Use Change Option #159

Open Nooshdokht-Bayatafshary opened 5 months ago

Nooshdokht-Bayatafshary commented 5 months ago

Dear Developers,

I am using the transient land use change option in my modeling and have a question about its functionality. I created a transient land use map with the following time steps:

My question is: If I set the start time for prerun modeling to 2015-01-01, will the model work correctly? Specifically, I want to know if the model will use the land use map from 2015-09-22 when needed, or if it will produce an error or return unrealistic results. According to the manual, it seems that the model loads the map at CalendarDayStart at the beginning of the simulation and replaces it at each subsequent time step with the map corresponding to that time step (or the most recent one available).

Thank you for your assistance!

Best regards, Nooshdokht

StefaniaGrimaldi commented 5 months ago

Dear @Nooshdokht-Bayatafshary,

thank you for your enquiry.

The model simulation will start at the "StepStart". The "CalendarDayStart" is the reference date for time depending inputs. At each computational time step, the option "TransientLandUseChange" will read and use the land use maps with the closest time stamp (https://github.com/ec-jrc/lisflood-code/blob/master/src/lisflood/hydrological_modules/landusechange.py#L106). In your example, land use maps for 2015-09-22 will be used at the time step 2015-01-01.

As a side note, your message mentions that the start date of the prerun is 2015-01-01. The adequate length of the prerun depends on climatic conditions and physical properties of the modelled area (mainly, soil properties). Therefore, there is no general rule to establish the minimum number of years. However, this page provides insights and recommendations https://ec-jrc.github.io/lisflood-code/3_step5_model-initialisation/.

We hope that our answer helps, kind regards, on behalf of the developers team, Stefania

Nooshdokht-Bayatafshary commented 4 months ago

Dear @StefaniaGrimaldi,

Thank you for your clarification and assistance. I have another question regarding this option, and I'm unsure whether to raise it in this section or the lisflood-calibration.

As I mentioned previously, I am using the transient land use change option in my modeling and I am currently trying to run calibration codes. I encountered an error that seems related to the transient land use change option. I provided the full error log log.txt and my XML settings (settings_calibrationPreRun0.zip).


Traceback (most recent call last):
  File "CAL_7A_CALIBRATION.py", line 68, in <module>
    calibrate_subcatchment(cfg, obsid, subcatch)
  File "CAL_7A_CALIBRATION.py", line 40, in calibrate_subcatchment
    model.init_run()
  File "/home/n.bayatafshary74.student.sharif/.conda/envs/lis3.8/lib/python3.8/site-packages/liscal/hydro_model.py", line 68, in init_run
    lisf1.main(prerun_file, '-i')
  File "/home/n.bayatafshary74.student.sharif/.conda/envs/lis3.8/lib/python3.8/site-packages/lisflood/main.py", line 226, in main
    lisfloodexe(lissettings)
  File "/home/n.bayatafshary74.student.sharif/.conda/envs/lis3.8/lib/python3.8/site-packages/lisflood/main.py", line 90, in lisfloodexe
    Lisflood = LisfloodModel()
  File "/home/n.bayatafshary74.student.sharif/.conda/envs/lis3.8/lib/python3.8/site-packages/lisflood/Lisflood_initial.py", line 188, in __init__
    self.landusechange_module.initial()
  File "/home/n.bayatafshary74.student.sharif/.conda/envs/lis3.8/lib/python3.8/site-packages/lisflood/hydrological_modules/landusechange.py", line 82, in initial
    soil = OrderedDict([(name, loadmap(epic_settings.landuse_inputmap[epic_settings.vegetation_landuse[name]])) for name in self.var.prescribed_vegetation])
  File "/home/n.bayatafshary74.student.sharif/.conda/envs/lis3.8/lib/python3.8/site-packages/lisflood/hydrological_modules/landusechange.py", line 82, in <listcomp>
    soil = OrderedDict([(name, loadmap(epic_settings.landuse_inputmap[epic_settings.vegetation_landuse[name]])) for name in self.var.prescribed_vegetation])
  File "/home/n.bayatafshary74.student.s
```harif/.conda/envs/lis3.8/lib/python3.8/site-packages/lisflood/global_modules/add1.py", line 332, in loadmap
    data = loadmap_base(*args, **kwargs)
  File "/home/n.bayatafshary74.student.sharif/.conda/envs/lis3.8/lib/python3.8/site-packages/lisflood/global_modules/add1.py", line 476, in loadmap_base
    raise LisfloodError(msg)
lisflood.global_modules.errors.LisfloodError:

 ========================== LISFLOOD ERROR =============================
time step -264 is not stored in ../Karkheh/outcal/CATCHMENTS_DIR/21413/maps/fracother.nc

Upon tracing the error, I believe it is due to the loadmap function using the default option of timestampFlag, which is set to "exact" (line 341 and line 129).

Could you please help me resolve this error? If this issue should be raised in the calibration section, please let me know. Thank you for your time and all your previous help.

Best regards, Nooshdokht

StefaniaGrimaldi commented 4 months ago

Dear @Nooshdokht-Bayatafshary,

@doc78 and I looked at your issue: we recommend setting <textvar name="MapsCaching" value="True"/>, as shown in the example settings files of the calibration tool (e.g. https://github.com/ec-jrc/lisflood-calibration/blob/master/integration/settings_EFAS1arcmin_referencetemplate.xml).

<textvar name="MapsCaching" value="True"/> will allow you to reduce the computational time in calibration and also to conveniently use the timestampFlag "closest" within the TransientLandUseChange module. Clearly, please do not hesitate to let us know whether the error persists.

As a matter of fact, your message highlighted a weakness in the module TransientLandUseChange: the loadmap function in line 82 should make use of the timestampFlag "closest". We will test this solution off-line, and if successful, we will include it in the next operational release in order to make the code more robust. Thank you for sharing your issue!

Kind regards, on behalf of the developers team, Carlo and Stefania

Nooshdokht-Bayatafshary commented 4 months ago

Dear @StefaniaGrimaldi and @doc78,

Thank you for your prompt and detailed response. It's encouraging to hear that my message has helped identify a potential improvement in the loadmap function of the TransientLandUseChange module. I appreciate your commitment to enhancing the robustness of the code and look forward to seeing this update in the next operational release.

I appreciate your recommendation and have reviewed the example settings file at the provided link. I have updated all other options in my settings file (settings_calibration.zip) as recommended specifically MapsCaching, but the error persists. Additionally, I printed the kwargs parameter line 229. When it attempts to read the fracother file, it only prints the value ({'value': '../Karkheh/outcal/CATCHMENTS_DIR/21413/maps/fracother'}) and does not include any timestampflag key. Therefore, I believe it is still passing the exact option, and the error remains.

Best regards, Nooshdokht

StefaniaGrimaldi commented 4 months ago

Dear @Nooshdokht-Bayatafshary,

@doc78 and I have looked at this issue again and, unfortunately, you are correct: it is necessary to implement a small bug fix.

You will need to change lines 82 and 129 in https://github.com/ec-jrc/lisflood-code/blob/fe94c33b6bd9fe09fa7be43a6dd4fdc8a11f744c/src/lisflood/hydrological_modules/landusechange.py

Here is the bug fix: soil = OrderedDict([(name, loadmap(epic_settings.landuse_inputmap[epic_settings.vegetation_landuse[name]], timestampflag='closest')) for name in self.var.prescribed_vegetation])

This will solve the issue that you flagged.

However, we tested your set-up, and it is very likely that you will encounter another error immediately after, specifically, in line 87. The error in line 87 is due to the misalignment between the masksmall.map and the netcdf maps (e.g. fracother). Please see the screenshot in attachment (from QGIS) masksmall_fracother

We hope that our answer helps, kind regards, Carlo and Stefania

Nooshdokht-Bayatafshary commented 4 months ago

Dear @StefaniaGrimaldi and @doc78,

Thank you for your continuous support and assistance. The change you suggested successfully resolved the initial error. As you mentioned, I encountered another error at line 87, and you were correct in identifying an issue with my setup: the NC file was flipped.

Upon further examination of the source code, I realized that both the loadmap_base function and the readnetcdf function exclude NaN values (represented by -999999). I am wondering if this exclusion is based on the maskmap or not.

Could you please confirm whether creating the netcdf files based on the boundary limits of my region (such as a rectangle) and using the maskmap to show the actual boundary of the basin will work correctly? I believe that my current method of defining boundaries might be causing the issue, and I am considering defining the boundaries using a larger rectangle.

Kind regards, Nooshdokht

StefaniaGrimaldi commented 4 months ago

Dear @Nooshdokht-Bayatafshary,

LISFLOOD computations are performed for all the pixels with mask map value = 1. In all the static and dynamic implementation maps, NaN (or Fill Values such as -999999) are not tolerated in pixels where mask map value = 1.

We hope that our answer helps, kind regards, Carlo and Stefania