Timestep issue when running SuPy with 1min forcing data

MartinaFrid commented 2 years ago

When running SUEWS with input data at 60s resolution, the model timestep is set to 60s, and the output is set to 60s, the resulting output is given for every 30s. Same happens when running the model for a 5min timestep with input data at 5min resolution. The results are then given every 2.5min. The results of these half timesteps looks very strange, with extremely high values of longwave out .

The figure show results while running SUEWS with 5min input data, with a timestep of 5min and the output is set to be given at 5min but are given at every 2.5min. 5min_in_out

sunt05 commented 2 years ago

Hi @MartinaFrid, please provide the input dataset that be used to reproduce this issue. Thanks!

MartinaFrid commented 2 years ago

Hi @sunt05 Here is the 1min data and a 5min dummy data that was used for the plot. both of them are giving strange result while using 1min and 5min time steps, respectively.

SUberlin_2022_data_1.txt test_2017_data_dummy_5.txt

sunt05 commented 2 years ago

Please also include other input files, i.e. RunControl.nml, initial condition, and SUEWS setting files, so one can more easily reproduce the issue. Thanks.

megstretton commented 2 years ago

Hi @sunt05,

My understanding of this issue was that SUEWS does not run when the three timesteps in the RunControl file are set to be the same value, i.e., ResolutionFilesOut, Tstep and ResolutionFilesIn.

I've attached a file containing my run that shows this. The input dummy data here should be the same as the 5 minute data that Martina provided. I am not getting half-minute timesteps as in her issue above. I also find that the model does not run when setting Tstep > 300, shown in tests using 15 minute forcing data.

I've run the same inputs changing the RunControl parameters as below to test this:

For 5 minute input data: ResolutionFilesOut=60 Tstep=60 ResolutionFilesIn=300

ResolutionFilesOut=300 Tstep=60 ResolutionFilesIn=300

ResolutionFilesOut=300 Tstep=300 ResolutionFilesIn=300

And for the 15 minute input data for: ResolutionFilesOut=900 Tstep=900 ResolutionFilesIn=900

ResolutionFilesOut=900 Tstep=300 ResolutionFilesIn=900

Files: test_input.zip

sunt05 commented 2 years ago

Thanks @megstretton for the input data - if SUEWS cannot run, please provide the error info you encountered so we can look into the issue.

@MartinaFrid One thing I noticed in the input met data is the missing of header line - which may confuse the met reader in SUEWS - please correct this and try again

MartinaFrid commented 2 years ago

Hi @sunt05

I have updated the header of the met input data but the results still look the same. The figure below show the results for 1min data with ResolutionFilesOut=60 Tstep=60 ResolutionFilesIn=60

files: SUberlin_setup_run1.zip

sunt05 commented 2 years ago

So, is the issue about abnormal Lout or output timestamp? Please be accurate.

MartinaFrid commented 2 years ago

It's about the timestamp. The abnormal Lout occurs on the extra 30s output timestamp. Sorry if that was unclear

Hämta Outlook for Androidhttps://aka.ms/AAb9ysg

From: Ting Sun @.> Sent: Friday, July 29, 2022 10:07:02 AM To: UMEP-dev/SUEWS @.> Cc: Martina Frid @.>; Mention @.> Subject: Re: [UMEP-dev/SUEWS] Timestep issue when running SUEWS with 1min data (Issue UMEP-dev/SuPy#58)

So, is the issue about abnormal Lout or output timestamp? Please be accurate.

— Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FUMEP-dev%2FSUEWS%2Fissues%2F120%23issuecomment-1198997464&data=05%7C01%7Cm.a.h.frid%40pgr.reading.ac.uk%7C157cc14a0899456485d408da713952b3%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637946788256687597%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dHTXha9e4i6J4BU8OpFx%2F%2B5RIXacp44vDahyuGLFsLo%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAUKFWM7U5NGPHJRXJUFNXZ3VWOGKNANCNFSM54VIR7FA&data=05%7C01%7Cm.a.h.frid%40pgr.reading.ac.uk%7C157cc14a0899456485d408da713952b3%7C4ffa3bc4ecfc48c09080f5e43ff90e5f%7C0%7C0%7C637946788256687597%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aWYKm1v77oS9pC6pzrtR9xpyLwXomesLZNN8OegbL6w%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>

suegrimmond commented 2 years ago

The issue for both cases (MF, MS) are the model output timestep

sunt05 commented 2 years ago

@MartinaFrid thanks for providing the input data (which are always helpful to debug/reproduce an issue) but I cannot reproduce your issue - since you didn't mention the SUEWS version I assume that's the latest commit from the master branch - and SUEWS stopped right away when reading the met file.

As such, several further issues in the forcing file were found (might not be exhaustive):

invalid temporal specifiers: imin should range 0–59 (60 is invalid); similarly,it should do 0–23 (24 is invalid).
wind speed should be positive;
rainfall should be non-negative.

Users are encouraged to use UMEP pre-processor for preparing a met file for SUEWS: https://umep-docs.readthedocs.io/en/latest/pre-processor/Meteorological%20Data%20MetPreprocessor.html

Please fix the issues in the met file first; then use the latest commit of master branch to try again.

MartinaFrid commented 2 years ago

Hi Ting Yeah, sorry about the data.

I have updated the met file, using the UMEP pre-processor. And I downloaded the newest version of SUEWS. The results is still the same, but I am running it in Supy, which I guess might make a difference. I have tried to update the supy version but I am not sure if a managed to get that correct

SuPy versions
-------------
supy: 2021.11.22 
supy_driver: 2021a2

=================
SYSTEM DEPENDENCY
INSTALLED VERSIONS
------------------
commit           : 945c9ed766a61c7d2c0a7cbb251b6edebf9cb7d5
python           : 3.8.12.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
Version          : 10.0.19043

new files: SUberlin_setup_run2.zip

MartinaFrid commented 2 years ago

Progress have been made, versions of supy have been updated. (personal communication)

Some error still occur, with 30s timesteps.

input files: SUberlin_setup_run2.zip Script: script_supy.zip

sunt05 commented 2 years ago

Thanks for these files - now I can reproduce the issue, which is due to a bug in the resampling function of supy - a fix has been released.

UMEP-dev / SuPy

Timestep issue when running SuPy with 1min forcing data #58