search

WorldCereal / ewoc_l8

Pre processing of L8 data (SR and TIR)
Other
0 stars 0 forks source link

Global Production V2 - L8SR products management #86

Open savmickael opened 1 year ago

savmickael commented 1 year ago

Some tiles exhibits errors during the L8 processing due to the presence of L8SR products which are didn't provide the TIR bands (B10 one for example).

The list of products are provided here

If the processor is run

Production env:

savmickael commented 1 year ago

Normally the workplan will filter some of the case: https://github.com/WorldCereal/ewoc_work_plan/issues/127

savmickael commented 1 year ago

related to https://github.com/WorldCereal/ewoc_l8/issues/79

savmickael commented 1 year ago

@agrosu1978 a quick and dirty solution is to remove from the db the L8 products with the following pattern; LC08_L2SR_*_02_T2

mbattude-cs commented 1 year ago

A critical error is already raised when B10 is missing : See commit https://github.com/WorldCereal/ewoc_l8/pull/80/commits/317e91e0fa26db48eb2f33e5057034f3b42286e7

Still need to handle this case : "without option, the processor must detect the product is not compatible with tir processing and raise a warning. The SR part must be process without issue." For now the SR part is not processed

savmickael commented 1 year ago

A critical error is already raised when B10 is missing : See commit 317e91e

Still need to handle this case : "without option, the processor must detect the product is not compatible with tir processing and raise a warning. The SR part must be process without issue." For now the SR part is not processed

not a priority to support the case where L8SR is put as input without the right option

mbattude-cs commented 1 year ago

See commit https://github.com/WorldCereal/ewoc_l8/commit/88a5e5cbdf9720699ac91e869aa25a0611086f80 SR : 7 bands TIR : 2 bands

> ewoc_generate_l8_ard --only-sr 32NKK LC08_L2SR_189058_20211023_20211103_02_T2 Uploaded 7 tif files to bucket |s3://ewoc-ard-dev/0000_000_20221215T132021/OPTICAL/32/N/KK/2021/20211023/LC08_L1T_20211023T235959_18905802T1_32NKK

> ewoc_generate_l8_ard --only-sr-mask 32NKK LC08_L2SR_189058_20211023_20211103_02_T2 Uploaded 1 tif files to bucket |s3://ewoc-ard-dev/0000_000_20221215T132246/OPTICAL/32/N/KK/2021/20211023/LC08_L1T_20211023T235959_18905802T1_32NKK

> ewoc_generate_l8_ard --only-tir 32NKK LC08_L2SR_189058_20211023_20211103_02_T2 [2022-12-15 13:23:08] ERROR:ewoc_l8.l8_process:Band LC08_L2SR_189058_20211023_20211103_02_T2_ST_B10.TIF does not exist ! [2022-12-15 13:23:08] ERROR:ewoc_l8.cli:Error during L8 ARD generation: ['LC08_L2SR_189058_20211023_20211103_02_T2'] not download ! [2022-12-15 13:23:08] CRITICAL:ewoc_l8.cli:Error during L8 ARD generation: No L8 ARD on 32NKK for ['LC08_L2SR_189058_20211023_20211103_02_T2'] !

> ewoc_generate_l8_ard 32NKK LC08_L2SR_189058_20211023_20211103_02_T2 [2022-12-15 13:25:36] WARNING:ewoc_l8.l8_process:1 L2SR products are removed as they are not compatible with TIR processing [2022-12-15 13:25:36] WARNING:ewoc_l8.l8_process:1 L2SR products are removed as they are not compatible with TIR processing Uploaded 7 tif files to bucket |s3://ewoc-ard-dev/0000_000_20221215T132325/OPTICAL/32/N/KK/2021/20211023/LC08_L1T_20211023T235959_18905802T1_32NKK

savmickael commented 1 year ago

Add this test as unittest please