delft3dfm WARNINGS

[backeb]

When running the workflow on 2023-03-31 I get the following errors, but the data is still generated.

tttz_waq_0000.dia:** WARNING: Requested time was: 2023-03-17T00:01:00Z !
tttz_waq_0000.dia:** WARNING: Updating target failed, quantity='waterlevelbnd', itemId=      56
tttz_waq_0000.dia:** WARNING: Updating source failed, quantity='polytim_item', item=      57, location=
tttz_waq_0000.dia:** WARNING: Updating target failed, quantity='polytim_item', itemId=      57
tttz_waq_0000.dia:** WARNING: Updating source failed, quantity='WATERLEVELBND', item=      58, location=EAST2_0001
tttz_waq_0000.dia:** WARNING: Datablock end (new [forcing] block) has been prematurely reached.
tttz_waq_0000.dia:** WARNING:    File: input/steric_east2.bc, Location: EAST2_0001, Quantity: WATERLEVELBND
tttz_waq_0000.dia:** WARNING: Error while updating boundary forcing at time=258984060.000
tttz_waq_0001.dia:** WARNING: The keyword DtMassBalance in the Processes section is now replaced by MbaInterval in the Output section.
tttz_waq_0001.dia:** WARNING: MbaInterval was not specified, will use DtMassBalance for backwards compatibility.
tttz_waq_0001.dia:** WARNING: readMDUFile: [processes] procesdatabasefile=proc_def.dat was in file, but not used. Check possible typo.

tttz_waq_0001.dia:** WARNING: Requested time was: 2023-03-17T00:01:00Z !
tttz_waq_0001.dia:** WARNING: Updating target failed, quantity='waterlevelbnd', itemId=      56
tttz_waq_0001.dia:** WARNING: Updating source failed, quantity='polytim_item', item=      57, location=
tttz_waq_0001.dia:** WARNING: Updating target failed, quantity='polytim_item', itemId=      57
tttz_waq_0001.dia:** WARNING: Updating source failed, quantity='WATERLEVELBND', item=      58, location=EAST2_0001
tttz_waq_0001.dia:** WARNING: Datablock end (new [forcing] block) has been prematurely reached.
tttz_waq_0001.dia:** WARNING:    File: input/steric_east2.bc, Location: EAST2_0001, Quantity: WATERLEVELBND
tttz_waq_0001.dia:** WARNING: Error while updating boundary forcing at time=258984060.000
tttz_waq_0002.dia:** WARNING: The keyword DtMassBalance in the Processes section is now replaced by MbaInterval in the Output section.
tttz_waq_0002.dia:** WARNING: MbaInterval was not specified, will use DtMassBalance for backwards compatibility.
tttz_waq_0002.dia:** WARNING: readMDUFile: [processes] procesdatabasefile=proc_def.dat was in file, but not used. Check possible typo.
tttz_waq_0003.dia:** WARNING: The keyword DtMassBalance in the Processes section is now replaced by MbaInterval in the Output section.
tttz_waq_0003.dia:** WARNING: MbaInterval was not specified, will use DtMassBalance for backwards compatibility.
tttz_waq_0003.dia:** WARNING: readMDUFile: [processes] procesdatabasefile=proc_def.dat was in file, but not used. Check possible typo.
tttz_waq_0004.dia:** WARNING: The keyword DtMassBalance in the Processes section is now replaced by MbaInterval in the Output section.
tttz_waq_0004.dia:** WARNING: MbaInterval was not specified, will use DtMassBalance for backwards compatibility.
tttz_waq_0004.dia:** WARNING: readMDUFile: [processes] procesdatabasefile=proc_def.dat was in file, but not used. Check possible typo.

ttz_waq_0004.dia:** WARNING: Requested time was: 2023-03-17T00:01:00Z !
tttz_waq_0004.dia:** WARNING: Updating target failed, quantity='waterlevelbnd', itemId=      56
tttz_waq_0004.dia:** WARNING: Updating source failed, quantity='polytim_item', item=      57, location=
tttz_waq_0004.dia:** WARNING: Updating target failed, quantity='polytim_item', itemId=      57
tttz_waq_0004.dia:** WARNING: Updating source failed, quantity='WATERLEVELBND', item=      58, location=EAST2_0001
tttz_waq_0004.dia:** WARNING: Datablock end (new [forcing] block) has been prematurely reached.
tttz_waq_0004.dia:** WARNING:    File: input/steric_east2.bc, Location: EAST2_0001, Quantity: WATERLEVELBND
tttz_waq_0004.dia:** WARNING: Error while updating boundary forcing at time=258984060.000
tttz_waq_0005.dia:** WARNING: The keyword DtMassBalance in the Processes section is now replaced by MbaInterval in the Output section.
tttz_waq_0005.dia:** WARNING: MbaInterval was not specified, will use DtMassBalance for backwards compatibility.
tttz_waq_0005.dia:** WARNING: readMDUFile: [processes] procesdatabasefile=proc_def.dat was in file, but not used. Check possible typo.

tttz_waq_0005.dia:** WARNING: Requested time was: 2023-03-17T00:01:00Z !
tttz_waq_0005.dia:** WARNING: Updating target failed, quantity='waterlevelbnd', itemId=      56
tttz_waq_0005.dia:** WARNING: Updating source failed, quantity='polytim_item', item=      57, location=
tttz_waq_0005.dia:** WARNING: Updating target failed, quantity='polytim_item', itemId=      57
tttz_waq_0005.dia:** WARNING: Updating source failed, quantity='WATERLEVELBND', item=      58, location=EAST2_0001
tttz_waq_0005.dia:** WARNING: Datablock end (new [forcing] block) has been prematurely reached.
tttz_waq_0005.dia:** WARNING:    File: input/steric_east2.bc, Location: EAST2_0001, Quantity: WATERLEVELBND
tttz_waq_0005.dia:** WARNING: Error while updating boundary forcing at time=258984060.000
tttz_waq_0006.dia:** WARNING: The keyword DtMassBalance in the Processes section is now replaced by MbaInterval in the Output section.
tttz_waq_0006.dia:** WARNING: MbaInterval was not specified, will use DtMassBalance for backwards compatibility.
tttz_waq_0006.dia:** WARNING: readMDUFile: [processes] procesdatabasefile=proc_def.dat was in file, but not used. Check possible typo.
tttz_waq_0007.dia:** WARNING: The keyword DtMassBalance in the Processes section is now replaced by MbaInterval in the Output section.
tttz_waq_0007.dia:** WARNING: MbaInterval was not specified, will use DtMassBalance for backwards compatibility.
tttz_waq_0007.dia:** WARNING: readMDUFile: [processes] procesdatabasefile=proc_def.dat was in file, but not used. Check possible typo.
tttz_waq.dia:** WARNING: The keyword DtMassBalance in the Processes section is now replaced by MbaInterval in the Output section.
tttz_waq.dia:** WARNING: MbaInterval was not specified, will use DtMassBalance for backwards compatibility.
tttz_waq.dia:** WARNING: readMDUFile: [processes] procesdatabasefile=proc_def.dat was in file, but not used. Check possible typo.

cc @lorincmeszaros @arthurvd

[lorincmeszaros]

Great, congratulations!! Note that these are not errors but Warnings. Based on my experience with Delft3D modelling, getting a bunch of warnings is okay :) of course it is wise to check them but only Errors will kill the process. The original model cofigurator, Lauriane, should know most about these warnings. Given the scope of the exercise (running workflow rather than perfet results..) I would ignore them for the time being.

[backeb]

Looking at the map data:

In [2]: import xarray as xr
In [3]: ds = xr.open_dataset('tttz_waq_0007_map.nc')
In [6]: ds.time.data
Out[6]: 
array(['2023-03-16T12:00:00.000000000', '2023-03-17T00:00:00.000000000',
       '2023-03-17T12:00:00.000000000'], dtype='datetime64[ns]')

Only data until 2023-03-17T12:00:00 is created, but date_min and date_max in workflow/config.yml are 2023-03-16 and 2023-03-18, respectively. It should have generated data until 2023-03-18T12:00:00

When I did the run yesterday for date_min = 2023-03-15 and date_max = 2023-03-17, ds.time.data had more data.

In [21]: ds.time.data
Out[21]: 
array(['2023-03-15T12:00:00.000000000', '2023-03-16T00:00:00.000000000',
       '2023-03-16T12:00:00.000000000', '2023-03-17T00:00:00.000000000',
       '2023-03-17T12:00:00.000000000'], dtype='datetime64[ns]')

[backeb]

Seems like it didn't download cmems data for 18 March, will need to investigate that.

[backeb]

Running the download rules individually downloads the correct data: snakemake -R download_cmems_physics --cores 1 and snakemake -R download_cmems_biogeochemistry --cores 1 fetches data up to 18 March.

Will test snakemake --cores 1 all, to double check things aren't going wrong with the parallelisation.

cc @sebastian-luna-valero

[backeb]

snakemake --cores 1 all seemed to work without problems, retesting with snakemake --cores 4 all

[backeb]

Now it seems to work...