Bugfix: Fix Point2Grid's handling of the `-qc` option for ADP input files

[hsoh-u]

Describe the Problem

The point2grid was implemented based on the comment at https://github.com/dtcenter/MET/issues/1194#issuecomment-568000825. It looks like it's not the finalized logic. The new logic might be updated at the RT ticket, not github. Here are a new logics from Ho-Chun:

(1) If simply handling total AOD, we do not need to specify the -adp option for point2grid. In this case, the qc flag is read from the same AOD input file and -qc=0,1,2 means AOD quality is "high", "medium", and "low", respectively. I believe this part should remain the same.

(2) If a need to map Smoke/Dust AOD, additional information of the values of smoke/dust detection confidence need to be read from ADP file to determine the status of quality of mapped Smoke/Dust AOD. Because the corresponding value of smoke/dust detection confidence changed in the new algorithm, from your email, the conversion from detection confidence to quality flag may need to be updated to reflect the new values.

I think the logic of conversion needs to be reviewed to reflect the new definition of ADP detection confidence. I recall we made a table for the conversion to high, medium, and low quality Smoke/Dust AOD. The current rule maybe like

AOD qc flag=0 + ADP Smoke detection confidence=12 ==> high quality Smoke AOD. AOD qc flag=0 + ADP Smoke detection confidence=4 ==> medium quality Smoke AOD. AOD qc flag=0 + ADP Smoke detection confidence=0 ==> low quality Smoke AOD

AOD qc flag=1 + ADP Smoke detection confidence=12 ==> medium quality Smoke AOD. AOD qc flag=1 + ADP Smoke detection confidence=4 ==> medium quality Smoke AOD. AOD qc flag=1 + ADP Smoke detection confidence=0 ==> low quality Smoke AOD

AOD qc flag=2 + ADP Smoke detection confidence=12 ==> low quality Smoke AOD. AOD qc flag=2 + ADP Smoke detection confidence=4 ==> low quality Smoke AOD. AOD qc flag=2 + ADP Smoke detection confidence=0 ==> low quality Smoke AOD.

AOD qc flag=0 + ADP Dudt detection confidence=48 ==> high quality Dust AOD. AOD qc flag=0 + ADP Dust detection confidence=16 ==> medium quality Dust AOD. AOD qc flag=0 + ADP Dust detection confidence=0 ==> low quality Dust AOD

AOD qc flag=1 + ADP Dust detection confidence=48 ==> medium quality Dust AOD. AOD qc flag=1 + ADP Dust detection confidence=16 ==> medium quality Dust AOD. AOD qc flag=1 + ADP Dust detection confidence=0 ==> low quality Dust AOD

AOD qc flag=2 + ADP Dust detection confidence=48 ==> low quality Dust AOD. AOD qc flag=2 + ADP Dust detection confidence=16 ==> low quality Dust AOD. AOD qc flag=2 + ADP Dust detection confidence=0 ==> low quality Dust AOD.

If you found the conversion relationship in current is like what I described above, then

======== the new rule https://github.com/dtcenter/MET/issues/2853 ==========

AOD qc flag=0 + ADP Smoke detection confidence=0 ==> high quality Smoke AOD. AOD qc flag=0 + ADP Smoke detection confidence=4 ==> medium quality Smoke AOD. AOD qc flag=0 + ADP Smoke detection confidence=8 ==> low quality Smoke AOD

AOD qc flag=1 + ADP Smoke detection confidence=0 ==> medium quality Smoke AOD. AOD qc flag=1 + ADP Smoke detection confidence=4 ==> medium quality Smoke AOD. AOD qc flag=1 + ADP Smoke detection confidence=8 ==> low quality Smoke AOD

AOD qc flag=2 + ADP Smoke detection confidence=0 ==> low quality Smoke AOD. AOD qc flag=2 + ADP Smoke detection confidence=4 ==> low quality Smoke AOD. AOD qc flag=2 + ADP Smoke detection confidence=8 ==> low quality Smoke AOD.

AOD qc flag=0 + ADP Dudt detection confidence=0 ==> high quality Dust AOD. AOD qc flag=0 + ADP Dust detection confidence=16 ==> medium quality Dust AOD. AOD qc flag=0 + ADP Dust detection confidence=32 ==> low quality Dust AOD

AOD qc flag=1 + ADP Dust detection confidence=0==> medium quality Dust AOD. AOD qc flag=1 + ADP Dust detection confidence=16 ==> medium quality Dust AOD. AOD qc flag=1 + ADP Dust detection confidence=32 ==> low quality Dust AOD

AOD qc flag=2 + ADP Dust detection confidence=0==> low quality Dust AOD. AOD qc flag=2 + ADP Dust detection confidence=16 ==> low quality Dust AOD. AOD qc flag=2 + ADP Dust detection confidence=32 ==> low quality Dust AOD.

Ho-Chun

Expected Behavior

The flag value 3 matches with high, but not other flag values:

• flag value 3 after bitwise shifting: high
• flag value 1 after bitwise shifting: medium
• flag value 0 after bitwise shifting: low
• And the adjust quality level based on the AOD QC flags
  • AOD qc value=1: flag value 3 after bitwise shifting becomes medium
  • AOD qc value=2: flag value 3 and 1 after bitwise shifting becomes low

Environment

Describe your runtime environment: 1. Machine: (e.g. HPC name, Linux Workstation, Mac Laptop) 2. OS: (e.g. RedHat Linux, MacOS) 3. Software version number(s)

To Reproduce

Describe the steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' 4. See error Post relevant sample data following these instructions: https://dtcenter.org/community-code/model-evaluation-tools-met/met-help-desk#ftp

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

2700044 NOAA CPC

Define the Metadata

Assignee

• [ ] Select engineer(s) or no engineer required
• [ ] Select scientist(s) or no scientist required

Labels

• [ ] Review default alert labels
• [ ] Select component(s)
• [ ] Select priority
• [ ] Select requestor(s)

Milestone and Projects

• [ ] Select Milestone as the next bugfix version
• [ ] Select Coordinated METplus-X.Y Support project for support of the current coordinated release
• [ ] Select MET-X.Y.Z Development project for development toward the next official release

Define Related Issue(s)

Consider the impact to the other METplus components.

• [ ] METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

Bugfix Checklist

See the METplus Workflow for details.

• [ ] Complete the issue definition above, including the Time Estimate and Funding Source.
• [ ] Fork this repository or create a branch of main_\. Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
• [ ] Fix the bug and test your changes.
• [ ] Add/update log messages for easier debugging.
• [ ] Add/update unit tests.
• [ ] Add/update documentation.
• [ ] Push local changes to GitHub.
• [ ] Submit a pull request to merge into main_\. Pull request: bugfix <Issue Number> main_<Version> <Description>
• [ ] Define the pull request metadata, as permissions allow. Select: Reviewer(s) and Development issue Select: Milestone as the next bugfix version Select: Coordinated METplus-X.Y Support project for support of the current coordinated release
• [ ] Iterate until the reviewer(s) accept and merge your changes.
• [ ] Delete your fork or branch.
• [ ] Complete the steps above to fix the bug on the develop branch. Branch name: bugfix_<Issue Number>_develop_<Description> Pull request: bugfix <Issue Number> develop <Description> Select: Reviewer(s) and Development issue Select: Milestone as the next official version Select: MET-X.Y.Z Development project for development toward the next official release
• [ ] Close this issue.

[jprestop]

@hsoh-u I noticed this issue was closed (likely automatically with the merge of the PR into main). I'm thinking it needs to be reopened for the work in develop, unless that's already been done. Can you please confirm and reopen if necessary?

[hsoh-u]

Yes, the develop branch is not updated, yet. Need to check SonarQube findings.