pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.04k stars 287 forks source link

Add QC-based filtering to the VIIRS EDR AOD550 product #2714

Closed djhoese closed 5 months ago

djhoese commented 6 months ago

See https://github.com/ssec/polar2grid/issues/666 for more information. Basically the AOD ("AOD550") product from the VIIRS EDR files looks a little "not good" by default. @kathys and I would like to play around with filtering it based on the QCAll QC variable in the file.

@kathys I think as a starting point maybe you could point me to some data files of a good test case and I could generate images for all of the QC cases (0, 1, 2, 3 - default) and we see which looks best. I don't think it necessarily matters what Satpy uses in this PR as I've turned it off by default with the general practice Satpy has of not interfering with data coming from the files.

kathys commented 6 months ago

@djhoese. A good test data set is here: bumi:/data/users/kathys/test_data/viirs_edr/j01/summer/asci/.

See image here: https://github.com/ssec/polar2grid/issues/666

codecov[bot] commented 6 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (afb4049) 95.39% compared to head (9d8b2fd) 95.40%. Report is 5 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2714 +/- ## ======================================= Coverage 95.39% 95.40% ======================================= Files 371 371 Lines 52691 52749 +58 ======================================= + Hits 50265 50323 +58 Misses 2426 2426 ``` | [Flag](https://app.codecov.io/gh/pytroll/satpy/pull/2714/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | Coverage Δ | | |---|---|---| | [behaviourtests](https://app.codecov.io/gh/pytroll/satpy/pull/2714/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | `4.16% <0.00%> (-0.01%)` | :arrow_down: | | [unittests](https://app.codecov.io/gh/pytroll/satpy/pull/2714/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | `96.01% <100.00%> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

coveralls commented 6 months ago

Pull Request Test Coverage Report for Build 7469837169

Warning: This coverage report may be inaccurate.

We've detected an issue with your CI configuration that might affect the accuracy of this pull request's coverage report. To ensure accuracy in future PRs, please see these guidelines. A quick fix for this PR: rebase it; your next report should be accurate.


Totals Coverage Status
Change from base Build 7416976545: 0.005%
Covered Lines: 50449
Relevant Lines: 52572

💛 - Coveralls
djhoese commented 5 months ago

@mraspaud I commented on your question. Otherwise I think this is ready to go with the small fix I just committed.

djhoese commented 5 months ago

Polar2Grid PR: https://github.com/ssec/polar2grid/pull/674