Change StitchBlobs time parsing

[mariellep]

Use TimeObj to correctly calculate calendar values for StitchBlobs. Needs to be able to handle two kinds of time units:

• Hours/minutes/etc since (common date, usually "1800-01-01" or "1900-01-01"
• Hours/minutes/etc since (start of the file)

Current method likely only uses the time units from the first file in the sequence, resulting in wrong calendar date for the "hours since start of file" time units.

[MorganBennet1]

I am having an issue generating BlobStats that is perhaps related to this... BlobStats runs fine when not calling "out_fulltime", however I also want the full timestep as it appears this is required for the R scripts and BlobMetrics. Calling the argument --out_fulltime results in the following exception:

EXCEPTION (/home/conda/feedstock_root/build_artifacts/tempest-extremes_1679047198141/work/src/base/TimeObj.cpp, Line 589) ToShortString() only valid for Time::TypeFixed

The error is thrown when writing blob data for the 2nd year (regardless of which year I start the executable on). All processing prior to this worked well, largely following doi:10.5194/gmd-14-5023-2021

Is this related...and any tips or tricks to overcome?

Cheers

[paullric]

Hmm… Suspicious. I wasn’t able to reproduce on my end. Can you send a simple example that breaks?

On Sep 14, 2023, at 9:41 PM, MorganBennet1 @.***> wrote:

I am having an issue generating BlobStats that is perhaps related to this... BlobStats runs fine when not calling "out_fulltime", however I also want the full timestep as it appears this is required for the R scripts and BlobMetrics. Calling the argument --out_fulltime results in the following exception:

EXCEPTION (/home/conda/feedstock_root/build_artifacts/tempest-extremes_1679047198141/work/src/base/TimeObj.cpp, Line 589) ToShortString() only valid for Time::TypeFixed

The error is thrown when writing blob data for the 2nd year (regardless of which year I start the executable on). All processing prior to this worked well, largely following doi:10.5194/gmd-14-5023-2021

Is this related...and any tips or tricks to overcome?

Cheers

— Reply to this email directly, view it on GitHub https://github.com/ClimateGlobalChange/tempestextremes/issues/9#issuecomment-1720624542, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIVPZBWDOCBNIGEQHCCQHDX2PMBFANCNFSM4FYCQVWQ. You are receiving this because you are subscribed to this thread.

[MorganBennet1]

Thank you so much for the speedy reply! I have attached the example executable and two files that returns the error at my end. Also the resultant output: its the first time step into the next year as you will see.

Also perhaps relevant: this is on ERA5 data

Thank you so much for the help Example.zip

[MorganBennet1]

Following on from this: the ERA5 data was involved in some CDO pre-processing, so perhaps something has gone wrong with dates in there...I'll look into in the meantime

[paullric]

Just tried it out on my end and it ran without issue. Do you have the latest version installed from GitHub? Or are you using the conda-forge install?

The only oddity I saw about the data file was that the time:units had an extra “.0” at the end. You might want to try changing it to "hours since 1900-01-01 00:00:00"

On Sep 14, 2023, at 11:09 PM, MorganBennet1 @.***> wrote:

Following on from this: the ERA5 data was involved in some CDO pre-processing, so perhaps something has gone wrong with dates in there...I'll look into in the meantime

— Reply to this email directly, view it on GitHub https://github.com/ClimateGlobalChange/tempestextremes/issues/9#issuecomment-1720728895, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIVPZAMR4CQUTS7QRQHYJLX2PWJBANCNFSM4FYCQVWQ. You are receiving this because you commented.

[MorganBennet1]

Very interesting - thank you for trying. I am using the conda-forge install. As I was able to get one year out before the error popped up I just merged my .nc files into one and that overcome the issue. Sorry for not thinking of that sooner. Appreciate the help