Bias correction package needs an update to handle the ASCII data from AIRNOW

[KaiWang-NOAA]

This issue is opened to document that an updated bias-correction code package is needed for the AQMv7 implementation to handle the change of AIRNOW data format (from BUFR to ASCII/CSV). PSL is currently working on updating the code.

[KaiWang-NOAA]

The proposed updated code delivery from PSL will be by the end of March, 2023

[HaixiaLiu-NOAA]

Thank you, Kai, for the update on this issue.

On Mon, Mar 20, 2023 at 10:19 AM KaiWang-NOAA @.***> wrote:

The proposed updated code delivery from PSL will be by the end of March, 2023

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/AQM/issues/61#issuecomment-1476320712, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKXODWK6XD3H5HEY6QCO3CLW5BRM3ANCNFSM6AAAAAAUW5UWCQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Dr. Haixia Liu (She/Her)

Lynker at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2856

College Park, MD 20740

@. @.>

301-683-3632

[KaiWang-NOAA]

The updated code has been delivered by PSL on April 12, 2023. The work of testing the code on WCOSS2 and merging into the workflow is underway now.

[HaixiaLiu-NOAA]

Kai completed testing the PSL's bias correction package on WCOSS2 and will work with Chan-Hoo for integration into the workflow. He updated in an email on this issue (see below)

"I'm sending you all the files that are needed for the updated BC package in the workflow to deal with the AIRNOW csv/netcdf files.

1) The source codes are in /lfs/h2/emc/physics/noscrub/kai.wang/Bias_correction/code_aqmv7_implementation/version_04122023/aqm_bias_correct.2023.0411/aqm_bias_correct.fd and aqm_bias_interpolate.fd. They should replace those in sorc/AQM-utils/sorc/aqm_bias_correct.fd and sorc/AQM-utils/sorc/aqm_bias_correct.fd. I believe the same CMAKE files can be used for the compilation.

2) The library files are in /lfs/h2/emc/physics/noscrub/kai.wang/Bias_correction/code_aqmv7_implementation/version_04122023/aqm_bias_correct.2023.0411/lib. They should replace those in sorc/AQM-utils/lib/lib.

3) The new configure files for interpolation are in /lfs/h2/emc/physics/noscrub/kai.wang/Bias_correction/code_aqmv7_implementation/version_04122023/aqm_bias_correct.2023.0411/aqmv7/run.interp/update_config/config.interp*. They should replace those in sorc/AQM-utils/parm/bias_correction

4) The new configure files for bias correction are in /lfs/h2/emc/physics/noscrub/kai.wang/Bias_correction/code_aqmv7_implementation/version_04122023/aqm_bias_correct.2023.0411/aqmv7/run.bias/update_config/config.*. They should replace those in sorc/AQM-utils/parm/bias_correction

5) The new site files are in /lfs/h2/emc/physics/noscrub/kai.wang/Bias_correction/code_aqmv7_implementation/version_04122023/aqm_bias_correct.2023.0411/aqmv7/run.interp/update_config/sites.valid*. They should replace those in sorc/AQM-utils/parm/bias_correction

6) The updated exregional scripts can be found in /lfs/h2/emc/physics/noscrub/kai.wang/nwdev/packages/aqm.v7.0.46/scripts/exregional_bias_correction_o3.sh and exregional_bias_correction_pm25.sh

7) a new converter code folder needs to be created in sorc/AQM-utils/sorc/aqm_airnow_converter.fd. The original codes from /lfs/h2/emc/physics/noscrub/kai.wang/Bias_correction/airnow/csvtonetcdf/converter/convert_airnow_csv.2023.0206 should be put there

8) in 6), I added a placeholder for one additional script that I need your help to put into. See lines 133-135 in exregional_bias_correction_o3.sh. The script convert.airnow.sh can be found in /lfs/h2/emc/physics/noscrub/kai.wang/Bias_correction/airnow/csvtonetcdf/converter/convert_airnow_csv.2023.0206/convert.airnow.sh. This script will invoke the executable "convert_airnow_csv" generated in 7). It will read in the csv files (see lines 121-123 and 129-131 of exregional_bias_correction_o3.sh) and generate the netcdf files and put them into folders corresponding to lines 125-127. The netcdf files will replace bufr files (i.e., xx021 and xx031)."

[HaixiaLiu-NOAA]

@KaiWang-NOAA I will close this issue since PSL's new package has been integrated into the workflow successfully. Thank you and @chan-hoo coordinating to get this task done.