Open michelleharrold opened 1 month ago
Log message from the scorecard using METv11.1 data indicates there is an error: 2024-09-17T16:26:16,221 ERROR [-1] [addJsonRow] e.u.m.s.GraphicalOutputManager: WARNING: one of the fields contains '-'. The scorecard could be invalid!!!!!
This is traced to the edu.ucar.metviewer.scorecard.GraphicalOutputManager.java addJsonRow method. The model name is checked for the presence of more than one '-' and if that condition is met, raises a warning and then replaces '-' with whitespace.
This has been verified by a database query for data from METv10.x (mv_rrfs_href_spring2022) and METv11.1 (mv_hwt2024):
MET10.x data:
MariaDB [mv_rrfs_href_spring2022]> select distinct(model) from stat_header; +------------+ | model | +------------+ | HREF | | HREF_mean | | HREF_mem01 | | HREF_mem02 | | HREF_mem03 | | HREF_mem04 | | HREF_mem05 | | HREF_mem06 | | HREF_mem07 | | HREF_mem08 | | HREF_mem09 | | HREF_mem10 | | HREF_prob | | RRFS | | RRFS_mean | | RRFS_mem01 | | RRFS_mem02 | | RRFS_mem03 | | RRFS_mem04 | | RRFS_mem05 | | RRFS_mem06 | | RRFS_mem07 | | RRFS_mem08 | | RRFS_mem09 | | RRFS_mem10 | | RRFS_prob | +------------+
MET 11.1 data:
MariaDB [mv_hwt2024]> select distinct model from stat_header; +---------------------+ | model | +---------------------+ | GSL-MPAS05_mem000 | | HRRR_mem000 | | NSSL-MPAS-HN_mem000 | | NSSL-MPAS-HT_mem000 | | NSSL-MPAS-RT_mem000 | +---------------------+
The METv11.1 has model names with more than one '-' present. This is most likely causing inadvertent "mangling" of the DIFF_SIG model name in METv11.1 data (and most likely affecting the generation of correct/expected statistical results):
Note the absence of the minus sign between the two models for the METv11.1 data:
DIFF_SIG(Z2 HRRR_mem000 350000 TMP ME-Z2 NSSL-MPAS-HN_mem000 350000 TMP ME)
Note the '-' to indicate the minus sign in the METv10.x data (which is expected/correct):
METv10.x:
DIFF_SIG(Z10 HREF_mem04 360000 WIND BCRMSE-Z10 RRFS_mem01 360000 WIND BCRMSE)
@bikegeek -- wooot! Thanks for tracking that down so quickly. Going forward, would it be possible to have more than one '-' present, or should we change our model name? If it is possible to only have one '-' present, we might want to document that, so users are aware.
Thank you!
I think moving forward we should allow model names with more than one '-'.
On Fri, Oct 25, 2024 at 4:12 PM michelleharrold @.***> wrote:
@bikegeek https://github.com/bikegeek -- wooot! Thanks for tracking that down so quickly. Going forward, would it be possible to have more than one '-' present, or should we change our model name? If it is possible to only have one '-' present, we might want to document that, so users are aware.
Thank you!
— Reply to this email directly, view it on GitHub https://github.com/dtcenter/METviewer/issues/553#issuecomment-2438950810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4UJHRD2ASSXHHSLWTW2XDZ5K64FAVCNFSM6AAAAABQO5R366VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZYHE2TAOBRGA . You are receiving this because you were mentioned.Message ID: @.***>
The Error raised from the GraphicalOutputManager might not be the cause of the error, it might just be the check that indicates something is wrong. I've set up an IDE on seneca to step through the code to see where the DIFF_STAT statistics are being computed.
thanks @bikegeek -- if you need someone to help test, work through this, please let me or @briannen know!
After reloading a database that changed the model names with multiple dashes to have underscores instead, I was able to make a scorecard successfully! So this error does seem to be directly related to multiple dashes.
Replace italics below with details for this issue.
Describe the Problem
@briannen and I are working on applying interactive scorecards to MET output from the DTC RRFS Agile Framework Team. We are able to generate correct scorecards from older MET output (v10.1.1), but with more current (v11.1.0) MET output the scorecard does not generate correct output. A scorecard is created, but all cells are 'Not statistically relevant.'
Expected Behavior
Using a similar (but not the same, since the underlying MET output is different) scorecard XML, I would expect a correct scorecard with statistically relevant data for all cells with both METv10.1.1 and METv11.1.0 data.
Environment
Describe your runtime environment: Testing is taking place on Dakota. The scorecard script we are pointing to is: /d3/projects/METViewer/src_scorecard/apps/METviewer/bin/mv_scorecard.sh
A test XML using METv10.1.1 data is here: /d3/projects/RRFS/metv/interactive/scorecard_rrfs.xml (output: /d3/projects/RRFS/metv/interactive/scorecard_rrfs_sfc.png)
A test XML using METv11.1.0 data is here: /d3/projects/RRFS/metv/interactive/briannen/scorecard_test.xml (log: /d3/projects/RRFS/metv/interactive/briannen/test.log) (output: /d3/projects/RRFS/metv/interactive/briannen/scorecard_hwt_2mT.png)
To Reproduce
Describe the steps to reproduce the behavior: On dakota: /d3/projects/METViewer/src_scorecard/apps/METviewer/bin/mv_scorecard.sh /d3/projects/RRFS/metv/interactive/briannen/scorecard_test.xml
Relevant Deadlines
Ideally, we would like a fix by mid-November, but if not, we would like to have this fixed by Jan. for the Annual AMS meeting.
Funding Source
FY23 NOAA Base (2792533) [NOTE: Subject to change to NOAA FY24!]
Define the Metadata
Assignee
Labels
Milestone and Projects
Define Related Issue(s)
Consider the impact to the other METplus components.
Bugfix Checklist
See the METplus Workflow for details.
bugfix_<Issue Number>_main_<Version>_<Description>
bugfix <Issue Number> main_<Version> <Description>
bugfix_<Issue Number>_develop_<Description>
Pull request:bugfix <Issue Number> develop <Description>
Select: Reviewer(s) and Development issue Select: Milestone as the next official version Select: METviewer-X.Y.Z Development project for development toward the next official release