gerritholl
commented
4 years ago @mraspaud said on slack that @adybbroe has seen this bug, but I could not find it in the issues list. My apologies if this is a duplicate.
Closed gerritholl closed 4 years ago
gerritholl
commented
4 years ago @mraspaud said on slack that @adybbroe has seen this bug, but I could not find it in the issues list. My apologies if this is a duplicate.
djhoese
commented
4 years ago While I acknowledge this is a real problem, I'm curious why you are loading these together in the same Scene? One quick workaround for the ABI versus NWCSAF-GEO composites is to load them separately. That doesn't help in the case of a reader that lists multiple sensors though.
gerritholl
commented
4 years ago I'm loading them together because the fogpy fls_day composite for ABI needs both:
fls_day:
compositor: !!python/name:fogpy.composites.FogCompositorDay
prerequisites:
- C02
- C03
- C05
- C07
- C11
- C14
- C15
- cmic_cot
- cmic_lwp
- cmic_reff
standard_name: fls_day
path_dem: /media/nas/x21308/DEM/dem_eu_1km.tifSo the not super-easy workaround:
scn_nwcsaf = Scene(...)
scn_nwcsaf.load(['cmic_cot', 'cmic_lwp', 'cmic_reff'])
scn_abi['cmic_cot'] = scn_nwcsaf['cmic_cot']
scn_abi['cmic_lwp'] = scn_nwcsaf['cmic_lwp']
scn_abi['cmic_reff'] = scn_nwcsaf['cmic_reff']
scn_abi.load(['fls_day'])
pnuu
commented
4 years ago I'm curious why you are loading these together in the same Scene?
This is also needed for the MaskingCompositor, where the data really need to be loaded at the same time.
adybbroe
commented
4 years ago I'm curious why you are loading these together in the same Scene?
This is also needed for the
MaskingCompositor, where the data really need to be loaded at the same time.
Yes agree.
adybbroe
commented
4 years ago @mraspaud said on slack that @adybbroe has seen this bug, but I could not find it in the issues list. My apologies if this is a duplicate.
Correct. I didn't yet come to make the issue @gerritholl. I found the bug/issue while preparing my presentation for yesterday, so was too busy. Thanks for posting it. Sorry for discovering it again!
djhoese
commented
4 years ago We have talked in the past about possibly defining multi-sensor composites in these special cases like viirs-abi.yaml or something. Not sure that is the best solution but it is an idea.
As mentioned on slack, the correct fix for this particular case is to fix the nwcsaf-geo reader's sensor_names property to only return the sensor for the files provided.
adybbroe
commented
4 years ago https://github.com/pytroll/satpy/blob/e91b6c7b8cc54ede695784211e3cd81a0682a051/satpy/readers/nwcsaf_nc.py#L80-L92 @djhoese As far as I can see this is what is tried here?
djhoese
commented
4 years ago That won't work for this particular problem. The .sensor property you have here doesn't mean anything to the base reader in yaml_reader which is what's provide the Scene with the information used to load composites.
gerritholl
commented
4 years ago As far as fogpy is concerned, it's not actually a multi-sensor composite, but it's still a multi-reader composite. But multiple sensors on the same platform should also be possible now that not only LEO but also GEO satellites are starting to have a more diverse set of instruments.
Describe the bug
When I load both ABI data and NWCSAF-GEO data, satpy loads composites files from
abi.yaml,visir.yaml, andseviri.yaml(with #1111, alsoahi.yaml). When I subsequently try to load a composite such asoverview, depending on which composite file was loaded first, it either succeeds (ifabi.yamlwas loaded first) or fails (ifseviri.yamlwas loaded first).To Reproduce
Expected behavior
I expect that this code consistently executes without error messages.
Actual results Text output of actual results or error messages including full tracebacks if applicable.
Sometimes it fails, sometimes it succeeds. This appear to depend on the order in which the composites are loaded, which appears non-deterministic.
Example of when it succeeds (
abi.yamlloaded beforeseviri.yaml):Example of when it fails (
seviri.yamlloaded beforeabi.yaml):Environment Info:
Additional context
PR #1111 explicitly lists abi and ahi as sensors for which the NWCSAF-GEO can work, this does not solve the problem.