search

Global-Flood-Assessment / MoMProduction

Production setup for ModelofModels
1 stars 1 forks source link

Incident report: GFMS Flood_byStor_2022103103.bin incomplete data #38

Closed webgisdeveloper closed 1 year ago

webgisdeveloper commented 1 year ago 
Traceback (most recent call last):
  File "rasterio/_io.pyx", line 701, in rasterio._io.DatasetReaderBase._read
  File "rasterio/shim_rasterioex.pxi", line 162, in rasterio._shim.io_multi_band
  File "rasterio/_err.pyx", line 192, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_AppDefinedError: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.vrt, band 1: IReadBlock failed at X offset 9, Y offset 12: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 794: Failed to read scanline 794.
webgisdeveloper commented 1 year ago 
Traceback (most recent call last):
  File "rasterio/_io.pyx", line 701, in rasterio._io.DatasetReaderBase._read
  File "rasterio/shim_rasterioex.pxi", line 162, in rasterio._shim.io_multi_band
  File "rasterio/_err.pyx", line 192, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_AppDefinedError: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.vrt, band 1: IReadBlock failed at X offset 9, Y offset 12: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 791: Failed to read scanline 791.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/geogateway/MoMProduction/HWRF_MoM.py", line 708, in <module>
    main()   
  File "/home/geogateway/MoMProduction/HWRF_MoM.py", line 705, in main
    batchrun_HWRF_MoM()
  File "/home/geogateway/MoMProduction/HWRF_MoM.py", line 693, in batchrun_HWRF_MoM
    update_HWRF_MoM(testdate)
  File "/home/geogateway/MoMProduction/HWRF_MoM.py", line 62, in update_HWRF_MoM
    GFMS_fixdate(adate[:8])
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 469, in GFMS_fixdate
    GFMS_processing(processing_dates)
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 392, in GFMS_processing
    GFMS_data_extractor(bin_file)
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 341, in GFMS_data_extractor
    GFMS_extract_by_watershed(vrt_file)
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 308, in GFMS_extract_by_watershed
    data_points = GFMS_extract_by_mask(vrt_file, test_json)
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 236, in GFMS_extract_by_mask
    out_image, out_transform = mask(src, [mask_json['features'][0]['geometry']], crop=True)
  File "/home/geogateway/miniconda3/envs/mom/lib/python3.9/site-packages/rasterio/mask.py", line 189, in mask
    out_image = dataset.read(
  File "rasterio/_io.pyx", line 367, in rasterio._io.DatasetReaderBase.read
  File "rasterio/_io.pyx", line 704, in rasterio._io.DatasetReaderBase._read
rasterio.errors.RasterioIOError: Read or write failed. /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.vrt, band 1: IReadBlock failed at X offset 9, Y offset 12: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 791: Failed to read scanline 791.
webgisdeveloper commented 1 year ago

Data for debug: 2022103103.zip

Flood_byStor_2022103103.csv Flood_byStor_2022103103.vrt Flood_byStor_2022103103.bin Flood_byStor_2022103103.hdr

webgisdeveloper commented 1 year ago

error when generate gfms tiff image

Input file size is 4916, 1600
0...10...20...30...40...50...60...70...80..ERROR 3: Failed to read scanline 791.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 791: Failed to read scanline 791.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.vrt, band 1: IReadBlock failed at X offset 0, Y offset 12: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 791: Failed to read scanline 791.
Input file size is 4916, 1600
webgisdeveloper commented 1 year ago

09 is also failed

0...10...20...30...40.ERROR 3: Failed to read scanline 418.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103109.bin, band 1: IReadBlock failed at X offset 0, Y offset 418: Failed to read scanline 418.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103109.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103109.bin, band 1: IReadBlock failed at X offset 0, Y offset 418: Failed to read scanline 418.
webgisdeveloper commented 1 year ago

12 is also failed

0...10...20...30...40.ERROR 3: Failed to read scanline 419.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103112.bin, band 1: IReadBlock failed at X offset 0, Y offset 419: Failed to read scanline 419.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103112.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103112.bin, band 1: IReadBlock failed at X offset 0, Y offset 419: Failed to read scanline 419.
webgisdeveloper commented 1 year ago

15 is also failed 

0...10...20...30...40.ERROR 3: Failed to read scanline 408.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103115.bin, band 1: IReadBlock failed at X offset 0, Y offset 408: Failed to read scanline 408.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103115.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103115.bin, band 1: IReadBlock failed at X offset 0, Y offset 408: Failed to read scanline 408.
webgisdeveloper commented 1 year ago

18

0...10...20...30...40.ERROR 3: Failed to read scanline 409.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103118.bin, band 1: IReadBlock failed at X offset 0, Y offset 409: Failed to read scanline 409.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103118.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103118.bin, band 1: IReadBlock failed at X offset 0, Y offset 409: Failed to read scanline 409.
webgisdeveloper commented 1 year ago

21

Input file size is 4916, 1600
0...10...20...30...40.ERROR 3: Failed to read scanline 407.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103121.bin, band 1: IReadBlock failed at X offset 0, Y offset 407: Failed to read scanline 407.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103121.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103121.bin, band 1: IReadBlock failed at X offset 0, Y offset 407: Failed to read scanline 407.