search

NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)
https://global-workflow.readthedocs.io/en/latest
GNU Lesser General Public License v3.0
75 stars 168 forks source link

Break vrfy job into separate jobs and retire from system #235

Closed KateFriedman-NOAA closed 10 months ago

KateFriedman-NOAA commented 3 years ago

The vrfy job has been a catch-all for verification and small downstream jobs for a long time. With the introduction of METplus verification in a separate job and the eventual retirement of the VSDB package the "vrfy" job begins to further lose its meaning.

Consider breaking this job into multiple jobs based on it's current sub-tasks:

Use existing switches in configs to determine whether to add jobs to workflow. Add new switches as needed.

RussTreadon-NOAA commented 3 years ago

Do we need to maintain1 and 4 in the future? It's my impression that we are transitioning away from VSDB and it's associated verification with the close of v16. METplus verification will take over these functions.

It's not clear to me if/how Fit2Obs (3) will interact/merge with METPlus.

GFS v15 already runs RadMon (5) and MinMon (7) as unique j-jobs. NCO's GFS v16 parallel runs RadMon (5), OznMon (6), and MinMon (7) as unique j-jobs.

KateFriedman-NOAA commented 3 years ago

Do we need to maintain1 and 4 in the future? It's my impression that we are transitioning away from VSDB and it's associated verification with the close of v16. METplus verification will take over these functions.

Items 1 and 4 will indeed go away (retire) in the near future but @yangfanglin has requested we keep VSDB supported until at least METplus provides graphical support (@malloryprow). Transition away from VSDB is still in progress.

It's not clear to me if/how Fit2Obs (3) will interact/merge with METPlus.

It's not clear to me yet either, I also await guidance from Jack/Mallory on this.

GFS v15 already runs RadMon (5) and MinMon (7) as unique j-jobs. NCO's GFS v16 parallel runs RadMon (5), OznMon (6), and MinMon (7) as unique j-jobs.

Roger that, those three can now be broken out of vrfy and put in their own separate jobs (using their JJOB scripts) in the dev mode (rocoto). Will do so.

cyclone track verification cyclone genesis verification cyclone genesis verification (FSU)

Question for all...put tracker/genesis in one job for all three pieces or separate jobs? Check how they run in ops.

malloryprow commented 3 years ago

@KateFriedman-NOAA we currently have support for the graphics. I don't know much about fit2obs, so I would like to see what Jack says about this.

yangfanglin commented 3 years ago

To replace VSDB with METplus, an offline diver/package outside of the workflow needs to be provided for all users to regenerate stats, make graphics and set up web displays. The offline driver needs to be supported on both operational and RD platforms.

Adding too many small jobs to the workflow makes the system rather messy. I'd suggest keeping items 1 to 4 together as one job and 8 to10 as another job.

Fanglin

On Tue, Jan 12, 2021 at 10:26 AM RussTreadon-NOAA notifications@github.com wrote:

Do we need to maintain1 and 4 in the future? It's my impression that we are transitioning away from VSDB and it's associated verification with the close of v16. METplus verification will take over these functions.

It's not clear to me if/how Fit2Obs (3) will interact/merge with METPlus.

GFS v15 already runs RadMon (5) and MinMon (7) as unique j-jobs. NCO's GFS v16 parallel runs RadMon (5), OznMon (6), and MinMon (7) as unique j-jobs.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/235#issuecomment-758730398, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKY5N2MK7ZFQNVCHTRK3VRLSZRSYLANCNFSM4V7LHGUQ .

-- Fanglin Yang, Ph.D. Chief, Model Physics Group Modeling and Data Assimilation Branch

NOAA/NWS/NCEP Environmental Modeling Center

https://www.emc.ncep.noaa.gov/gmb/wx24fy/fyang/ https://www.emc.ncep.noaa.gov/gmb/wx24fy/fyang/

malloryprow commented 3 years ago

@yangfanglin Yup we have all that. EMC_verif-global runs outsides the workflow. It has been how I have been producing daily stats for operational global models and their graphics as well as the graphics for the GFSv16 retrospective and real-time webpages.

yangfanglin commented 3 years ago

Mallory, That is great. Do you have a user guide to share with the developers ?

On Tue, Jan 12, 2021 at 10:47 AM malloryprow notifications@github.com wrote:

@yangfanglin https://github.com/yangfanglin Yup we have all that. EMC_verif-global runs outsides the workflow. It has been how I have been producing daily stats for operational global models and their graphics as well as the graphics for the GFSv16 retrospective and real-time webpages.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/235#issuecomment-758747540, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKY5N2MGDB5AA7HY4VRF3STSZRVKJANCNFSM4V7LHGUQ .

-- Fanglin Yang, Ph.D. Chief, Model Physics Group Modeling and Data Assimilation Branch

NOAA/NWS/NCEP Environmental Modeling Center

https://www.emc.ncep.noaa.gov/gmb/wx24fy/fyang/ https://www.emc.ncep.noaa.gov/gmb/wx24fy/fyang/

KateFriedman-NOAA commented 3 years ago

@KateFriedman-NOAA we currently have support for the graphics. I don't know much about fit2obs, so I would like to see what Jack says about this.

@malloryprow Oh excellent...ok it's the offline driver support that we need rather (as Fanglin stated). Do you or the MET team have anything like that yet or in the works?

Adding too many small jobs to the workflow makes the system rather messy. I'd suggest keeping items 1 to 4 together as one job and 8 to10 as another job.

@yangfanglin I agree, thanks for your feedback! With your suggested breakout the vrfy job would lose items 5-10 and become five separate jobs:

  1. vrfy = items 1-4 (grib1, MOS, Fit2Obs, VSDB)
  2. trak = items 8-10 (tracker and 2x genesis)
  3. radmon
  4. ozmon
  5. minmon

Suggested job names listed...anyone have a better suggestion for the "trak" job name?

Add new configs for new jobs as needed.

malloryprow commented 3 years ago

@yangfanglin I typed up a README at https://github.com/NOAA-EMC/EMC_verif-global. I'd be happy to type up or create a website as more detailed guide, if people feel it is needed.

lgannoaa commented 2 years ago

Hi @KateFriedman-NOAA and @malloryprow, Is this issue still need help? I can take a look and work on it.

KateFriedman-NOAA commented 2 years ago

@lgannoaa Yes, this issue is still desired to get completed. Please review the comments in the issue above and let us know what you would plan to do to resolve it before starting anything. This will require, at least, modifying the setup scripts to add new jobs for whatever gets pulled out of the main vrfy job and adding scripts/configs for the new jobs. Please write up a short summary of your plan in this issue and code managers will give their feedback. Thanks!