Adding new Feature/timestep

[drnimbusrain]

This PR addresses adding the capability for canopy-app to read/process/write multiple time steps, rather than only instantaneous single hour.

This adds multiple input NC and TXT files to the input/ directory for examples in the SE domain, where by default the output is put into single NC file with multiple time steps for NC file output. For TXT output, canopy-app sends the output to multiple txt files with included valid timestamp on the filename.

Here is an example for input downward shortwave flux and 2-m temperature, and output summed ISOP emissions valid @ July 01, 2022 at 11, 12, and 13 UTC:

Input Downward Shortwave Flux: July 01 @ 11 UTC July 01 @ 12 UTC July 01 @ 13 UTC Input 2-m Temperature July 01 @ 11 UTC July 01 @ 12 UTC July 01 @ 13 UTC Output Summed ISOP Biogenic Emissions July 01 @ 11 UTC July 01 @ 12 UTC July 01 @ 13 UTC

@zmoon Since the new SE example files cover three time periods, I assume this will break the current CI?

[drnimbusrain]

@zmoon Know your super busy, but maybe start a review of this next week ?

Thank you!

[drnimbusrain]

Starting playing with changes to NL for CI and the python/canopy_app.py script. However, still brakes the python testing as it doesn't like the new time suffixes placed on the end of the txt files it seems. @zmoon Any suggestions?

[quaz115]

Starting playing with changes to NL for CI and the python/canopy_app.py script. However, still brakes the python testing as it doesn't like the new time suffixes placed on the end of the txt files it seems. @zmoon Any suggestions?

Are the latest changes for passing CI/CD tests?

[drnimbusrain]

Starting playing with changes to NL for CI and the python/canopy_app.py script. However, still brakes the python testing as it doesn't like the new time suffixes placed on the end of the txt files it seems. @zmoon Any suggestions?

Are the latest changes for passing CI/CD tests?

@quaz115 Yes, but still unsuccessful, would like @zmoon help here as he put the tests in originally. Its something in the python/canopy_app.py script with my new time stamp suffixes put on the output files here. Zach please make any necessary adjustments to CI python scripts directly to make this work. Thank you!

[zmoon]

@drnimbusrain I fixed the output file recognition, but haven't dealt with possible multiple files/times yet.

I am not a huge fan of having the whole time step in the file name. And : aren't allowed in paths on Windows which could be a pain. Two possible solutions:

1. Simplified time stamp, e.g. 2022-07-01-12:00:00.0000 -> 2022-07-01_12 (or 2022-07-01_12-00 or such if need to have minute capability)
2. Have the automatic part just be integer step relative to run start (t000, t001, etc.), while user can still control the prefix with file_out nl setting. The time is already in the nc files. For the text files, could add a line like the reference height one.

[drnimbusrain]

@drnimbusrain I fixed the output file recognition, but haven't dealt with possible multiple files/times yet.

I am not a huge fan of having the whole time step in the file name. And : aren't allowed in paths on Windows which could be a pain. Two possible solutions:

1. Simplified time stamp, e.g. 2022-07-01-12:00:00.0000 -> 2022-07-01_12 (or 2022-07-01_12-00 or such if need to have minute capability)
2. Have the automatic part just be integer step relative to run start (t000, t001, etc.), while user can still control the prefix with file_out nl setting. The time is already in the nc files. For the text files, could add a line like the reference height one.

@zmoon Given you good suggestions, I decided to go with your option 2 for the reasons you described. Hopefully we can move this PR forward. Thanks again!

[drnimbusrain]

@zmoon Can We move this forward, would like to get this capability into canopy-app soon for testing other updates. Thanks!

[zmoon]

@drnimbusrain how should I modify the default namelist to test the multiple timestep txt output?

[drnimbusrain]

@drnimbusrain how should I modify the default namelist to test the multiple timestep txt output?

@zmoon Maybe I misunderstand your question, but you can look at how to simply run the default three timesteps for NC or TXT file in the branches updated namelist: https://github.com/noaa-oar-arl/canopy-app/blob/feature/timestep/input/namelist.canopy

[zmoon]

Like this commented line (seems like the default is still one time step)?

https://github.com/noaa-oar-arl/canopy-app/blob/33e90682cea11c089608637b22e4992668799586/input/namelist.canopy#L7

Was kind of hoping for point example.

[drnimbusrain]

Like this commented line (seems like the default is still one time step)?

https://github.com/noaa-oar-arl/canopy-app/blob/33e90682cea11c089608637b22e4992668799586/input/namelist.canopy#L7

Was kind of hoping for point example.

Oh, I just commented that out and reverted to single timestep to see if it would help the current CI pass for now. The line commented out is indeed the example for hopefully the new default three timestep example:

file_vars    = 'input/gfs.t12z.20220630.sfcf023.canopy.txt' 'input/gfs.t12z.20220701.sfcf000.canopy.txt' 'input/gfs.t12z.20220701.sfcf001.canopy.txt'

With other related NL text settings for the three times to match:

  infmt_opt   =  1
  time_start  = '2022-07-01-11:00:00.0000'
  time_end    = '2022-07-01-13:00:00.0000'
  ntime       =  3
  time_intvl  =  3600

Should have explanations of these new NL time variables in the README.

[drnimbusrain]

@zmoon But I see now what you are asking. I reverted back to the three times as default NL for your CI updates here, and provide an example point file for three times as well in latest commit. Let me know if this works for your changes/testing. Note I change the example point input file names as well.

[drnimbusrain]

@zmoon

@quaz115 is wondering how this PR is going, so he can merge and test with the updated biogenics and leafage, with some aspect of saving previous times (for LAI inputs).