Closed pnuu closed 4 years ago
Variable transparency using cloud type data
Some ideas....
Mask a dataset with another dataset (nitrogendioxide_tropospheric_column
with [qa_value
] , e.g. S5P Tropomi NO2 data)
masking is possible with:
scn[
nitrogendioxide_tropospheric_column] = scn[
nitrogendioxide_tropospheric_column].where(scn['qa_value‘]>0.75)
First pic colorized NO2 with quality flag applied, second pic the qa_dataset itself.
.yaml could look like:
no2_tropospheric_clean:
compositor: !!python/name:satpy.composites.ValueMaskCompositor
prerequisites:
- nitrogendioxide_tropospheric_column
- qa_value
modifiers: >0.75
standard_name: no2_tropospheric_clean
Similar approach for cutting unwanted data, e.g. from h03B data. This means to change a dateset by cutting data at the end or top. For h03B and to get rid of not processed data I need a
new_scene[
h03B] = new_scene[
h03B].where(new_scene[
h03B] > 0.0001388)
Some examples: raw, with dataset cut and colorized the cutted dataset:
this modified dataset I want to colorize and then use the BackgroundCompositor to add it on top of e.g. IR108.
.yaml could look like:
colorized_IRR:
compositor: !!python/name:satpy.composites.ValueMaskCompositor
prerequisites:
- h03B
modifiers: >0.0001388
standard_name: colorized_IRR
Maybe it could be the same compositor. If I provide a second dataset with a modifier, the second dataset is used to mask the first with optional conditions (<,> ==<...). If there was only one dataset given with a modifier, the datastet itself is masked by the value given. But maybe there is no universal approach possible.
As you pointed, maybe a modifier could be added at the Background compositor to give set an alpha value for the dataset which will place on top. Next step could be a variable alpha set by the data of the on top placed dataset.
Another idea: a common modifier which can do some easy math (...*3600
) for the channel, like this:
attrs = scn['h03B'].attrs
scn['h03B''] = 3600*scn['h03B']
scn['h03B''].attrs = attrs
scn['h03B'].attrs['units'] = r'mm/hr'
But maybe this is better applied inside the enhancements .yaml as there it would be more useful to get handy values for colorize...
Thank you @peters77! The .where()
case is a good one, but I think the YAML would look more like this:
modifiers:
n02_mask:
compositor: !!python/name:satpy.composites.MaskModifier
conditions:
# The conditions to apply, with the associated transparency
- "> 0.75": 100.0
prerequisites:
- qa_value
composites:
no2_tropospheric_clean:
compositor: !!python/name:satpy.composites.ValueMaskCompositor
prerequisites:
- name: nitrogendioxide_tropospheric_column
modifiers: [no2_mask]
standard_name: no2_tropospheric_clean
The last (multiplication and unit assignment) isn't masking and would need another feature issue. It should be pretty simple to implement as an enhancement or modifier. Not sure which would be more .
Yes, that looks better. What will this "100" mean in
modifiers:
n02_mask:
compositor: !!python/name:satpy.composites.MaskModifier
conditions:
# The conditions to apply, with the associated transparency
- "> 0.75": 100.0
- prerequisites:
qa_value
Do you intend to make a mask of the "qa_value" and the set the alpha to 100% to mask the bad data out!? So you somone is able to do even a blend of bad data!? Interesting!
Would your proposed idea even cover this:
new_scene[h03B] = new_scene[h03B].where(new_scene[h03B] > 0.0001388)
colorized_IRR:
compositor: !!python/name:satpy.composites.ValueMaskCompositor
prerequisites:
- h03B
modifiers: >0.0001388
standard_name: colorized_IRR
as here I didn't mask with another dataset but just with a "cutted" version of the dataset itself? Yes..the last on was just something what raised now 2 two times while colorize....maybe another modifier but does not fit into this masking addon..
What will this "100" mean
It's the transparency used for the condition. This way we could use the same modifier structure in a more flexible way.
Would your proposed idea even cover this:
new_scene[h03B] = new_scene[h03B].where(new_scene[h03B] > 0.0001388)
Yes, just define a modifier using the h03B
dataset:
h03b_mask:
compositor: !!python/name:satpy.composites.MaskModifier
conditions:
# The conditions to apply, with the associated transparency
- "> 0.0001388": 100.0
prerequisites:
- name: h03B
That sounds great! 👍 Hope you will have time to work on or implement it at PCW fully.
Feature Request
Currently we have
MaskingCompositor
that can be used to mask data with categorical data from NWC SAF PPS and GEO Cloud Type data. There is alsoCloudCompositor
that does masking by creating the mask from the data it self (and adds some transitional smoothing for the transparency).We should have a more flexible and generic way of masking data, e.g. using quality flags, thresholds and even combination of several things.
Describe the solution you'd like A generic way for describing different masking operations in the composite YAML files. Either as a modifier, or as a compositor.
Describe any changes to existing user workflow This feature would simplify the workflow for masking data in multiple use cases.
Additional context There are two approaches from which to choose:
The compositor approach would mean "wrapping" all the desired composites within another compositor. The modifier should be much easier to apply.
But the thing I need input for (I'll add something below on a new comment):