ilianagenkova commented 2 years ago

To address RRFS model's observations needs, add a RRFS network to obsproc's capability, and generate desired bufr dump and prepbufr obs files.

Due December 2022

[x] Leverage off RAP network's scripts, ush scripts and triggers to set RRFS up and running (rrfs, rrfs_e, rrfs_p)
[x] Update RRFS to obsproc.v1.1.0 to generate gmi1cr, snocvr, saldrn and subpfl, add Sent-6 to gpsro, add Metop-B/C AMV to satwnd, add saldrn and subpfl to nsstbufr
[x] Split the large radiances dump group to speed up processing (for global, and may be later for rrfs)
[x] Replace -np xx with -np $NPROCS (see exrap_dump.sh as a good example) (IG)
[x] Replace mpiexec with launcher_DUMP (see exurma_dump.sh as a good example) (IG)
[x] Resolve BUFRLIB: CLOSBF warnings in dump jobs(http://www2.spa.ncep.noaa.gov/bugzilla/show_bug.cgi?id=1350) (IG - under testing)
[x] Unify time dump window - set _earliest to -1h, and _latest to +1h (confirm with @JacobCarley-NOAA ) ( /Utils/GO_CHKDAT (?)/go_chkdat dump_file_name)
[x] Generate CONUS 15min AMVs dumps (in separate "regional" satwnd dumps) (bufr-dump/fix/bufr_dumplist , satwhr.bufr_d, exrrfs_dump*.sh : 00505**) (SN - under testing & soon to add to Ashley's collecting branch
[x] Verify and maintain generation of Direct Broadcast radiances (es and db bufr dumps)
[x] Confirm all GPSRO observation streams - DO5, Eumetsat, any others?
[x] Confirm domain - see mask files

Add JPSSS-2 NOAA-21 products of interest. Due at NOAA-21 provisional readiness, early 2023

[x] L1B ATMS rads (IG - tanks are generated in NRT, need to start dumping)
[x] L1B CrISS rads (IG - tanks are generated in NRT, need to start dumping
[x] VIIRS (or AVHRR??) AMVs - (IG - waiting on NESDIS )
[ ] SST - (IG - waiting on NESDIS )
[ ] Land surface temperature - Need info from RRFS team , Waiting on NESDIS

Due March 2023 (May-June 2023 is more realistic)

[x] Add UPRAIR full resolution bufr dumps (need Chris Hill's T2B code to be completed)
[x] Add T2B prepbufr (need Chris Hill's T2B code to be completed)
[ x] Add VIIRSS Fire radiative power (RAVE product sample due March 2023)
[ x] Add GOES Fire radiative power (also in RAVE??)
[ x] Add RAVE (uses VIIRSS add GOES Fire radiative power as inputs, see emails Shobha, chat with Ravan)
[ x] Add AOD and ADP aerosol products - on PDA, format? Satingest or DF? see email Amanda
[ x] Add surface pm2.5 air constituents , smoke and dust NESDIS products - EPA AirNow - email Amanda and Hongli
[ x] Add GOES Lightning Mapper data (source data, format, routing, etc) - Amanda Black, NESDIS products, submit ticket to DF (data on LDM) see email with Amanda
[x] Add wind tower data - review tanks and update bufr_dump to include dumping it (bufr_dumplist, wndtow 002/020, 002/021, 000/021, exrrfs_dump.sh, add to smallest Dump group (least run time) )

ilianagenkova commented 2 years ago

Development Branch: https://github.com/NOAA-EMC/obsproc/tree/feature/RRFS.obsproc.addition

ilianagenkova commented 1 year ago

Wind Tower tanks: export TANK_000020=/lfs/h1/ops/dev/dcom/202304/b000/xx020 (wind energy nacelle) export TANK_002020=/lfs/h1/ops/dev/dcom/202304/b002/xx020 (wind energy tower) export TANK_002021=/lfs/h1/ops/dev/dcom/202304*/b002/xx021 (wind energy tower) Will not go into prepbufr, due to BUFR table differences.

ilianagenkova commented 1 year ago

Change jrrfs_*.wc2.pbs triggers : From:

PBS -A OBSPROC-DEV

PBS -q dev

To:

PBS -A RRFS-DEV

PBS -q pridev

Group: emc.lam Group space: /lfs/h2/emc/lam/noscrub/emc.lam

ilianagenkova commented 1 month ago

@ShunLiu-NOAA , this was the user request as to what to go into the RRFS observational network.

I will close this ticket, as we met all requirements' (from @JacobCarley-NOAA and @MatthewPyle-NOAA ). RRFS is going into obsproc v1.3 in production.

If the RRFS obs files need to be updated/refined, we can work on it with another issue.

MatthewPyle-NOAA commented 1 month ago

Thanks @ilianagenkova . Is there a known timeline for obsproc v1.3 going into production?

ilianagenkova commented 1 month ago

Late 2024 - early 2025

On Mon, Sep 23, 2024 at 7:38 AM MatthewPyle-NOAA @.***> wrote:

Thanks @ilianagenkova https://github.com/ilianagenkova . Is there a known timeline for obsproc v1.3 going into production?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/obsproc/issues/74#issuecomment-2367954682, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOC4YXVWEH2JQONRSHXJN33ZX74RPAVCNFSM6AAAAABOUX5RZGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRXHE2TINRYGI . You are receiving this because you were mentioned.Message ID: @.***>

ShunLiu-NOAA commented 1 month ago

Great! The timeline is good for RRFS implementation.

Thank you.

Shun

On Mon, Sep 23, 2024 at 11:37 AM iliana Genkova @.***> wrote:

Late 2024 - early 2025

On Mon, Sep 23, 2024 at 7:38 AM MatthewPyle-NOAA @.***> wrote:

Thanks @ilianagenkova https://github.com/ilianagenkova . Is there a known timeline for obsproc v1.3 going into production?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/obsproc/issues/74#issuecomment-2367954682,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AOC4YXVWEH2JQONRSHXJN33ZX74RPAVCNFSM6AAAAABOUX5RZGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRXHE2TINRYGI>

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/obsproc/issues/74#issuecomment-2368666461, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD5GZ2MGR4PQKAFEW6EZDLZYAYSVAVCNFSM6AAAAABOUX5RZGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRYGY3DMNBWGE . You are receiving this because you were mentioned.Message ID: @.***>

NOAA-EMC / obsproc

Add RRFS network to obsproc's capability #74

PBS -A OBSPROC-DEV

PBS -q dev

PBS -A RRFS-DEV

PBS -q pridev