Closed LiangJYu closed 1 year ago
Red curly brackets in pic below show where stats are in HDF5
It seems we are still generating the JSON file for the stats. Would it be possible to remove it?
Do you prefer:
Let's keep the ability of generating the JSON file but let's not produce it
also, shall we indicate that stats
are actually QA metric by calling them qa_stats
?
Let's keep the ability of generating the JSON file but let's not produce it
Currently the HDF5 doesn't include all the QA fields found in the JSON (pixel_classification_percentatges
, rfi_info
, orbit_info
).
What do you think about consolidating all QA items under one quality_assurance
group in the HDF5 as detailed below?
science/SENTINEL1/CSLC
├── corrections
├── grids
├── metadata
└── quality_assurance (NEW)
├── orbit_info
│ └── orbit_type
├── pixel_classification_percentages
│ ├── topo
│ │ ├── percent_layover_pixels
│ │ ├── percent_shadow_pixels
│ │ └── percent_combined_pixels
│ ├── percent_land_pixels
│ └── percent_valid_pixels
├── rfi_info
│ ├── is_rfi_info_available
│ ├── rfi_mitigation_performed
│ ├── rfi_mitigation_domain
│ └── rfi_burst_report
└── stats
├── corrections
│ └── <correction_name>
│ ├── mean
│ ├── min
│ ├── max
│ └── std
└── grids
└── <polarization>
├── mean
├── min
├── max
└── std
Yes, I like that. Be aware pixel_classification_percentages that this:
│ ├── topo
│ │ ├── percent_layover_pixels
│ │ ├── percent_shadow_pixels
│ │ └── percent_combined_pixels
should go into the static layers topo product and not in CSLC
Yes, I like that. Be aware pixel_classification_percentages that this:
│ ├── topo │ │ ├── percent_layover_pixels │ │ ├── percent_shadow_pixels │ │ └── percent_combined_pixels
should go into the static layers topo product and not in CSLC
For the sake of consistency, should I add quality_assurance
group to the static layer HDF5 with:
science/SENTINEL1/CSLC
science/SENTINEL1/CSLC
├── grids
├── metadata
└── quality_assurance (NEW)
├── orbit_info
│ └── orbit_type
├── pixel_classification_percentages
│ └── topo
│ ├── percent_layover_pixels
│ ├── percent_shadow_pixels
│ └── percent_combined_pixels
└── stats
└── static_layers
└── <layer_name>
├── mean
├── min
├── max
└── std
Absolutely, let's go for it.
A couple of comments:
let's spell orbit_info
with orbit_information
and pixel_classification_percentages
with pixel_classification
For the burst I processed, both the items in pixel_classification
are empty. Is this intentional?
This is what I see in rfi_info
(let's spell rfi_information
)
is_rfi_info_available Dataset {SCALAR}
Data:
TRUE
rfi_burst_report Dataset {SCALAR}
Data:
"\000"
rfi_mitigation_domain Dataset {SCALAR}
Data:
TRUE
rfi_mitigation_performed Dataset {SCALAR}
Data:
TRUE
Is this intentional? For the RFI domain I was expecting a string.
Let's spell stats
with statistics and imag
with imaginary
pixel_classification
for both CSLC and static layers are 0 (I assume that's what you mean by empty) because it's a placeholder field with no computation being done to populate.rfi_domain
should be a string. Since it's a placeholder, I've switch to an empty string as a placeholder value.
This PR: