search

NOAA-OWP / inundation-mapping

Flood inundation mapping and evaluation software configured to work with U.S. National Water Model.
Other
92 stars 27 forks source link

[8pt] Urgent: test_case_by_hydro_id.py created small number of features #1258

Closed RobHanna-NOAA closed 7 hours ago

RobHanna-NOAA commented 3 weeks ago

In the current FIM 4.5.2.11 / HV 2.1.8 release we are loading currently into HV, we discovered a record count issue coming from test_case_by_hydro_id.py.

When we ran it last in the summer 2023 as part of FIM 4.4.0.0, it created a fim_performance_catchments.csv with 990,941 records. The newest edition ran a few weeks ago created just 9,451. This needs to be research.

The command I ran was: python foss_fim/tools/test_case_by_hydro_id.py -v fim_4_5_2_11 -g /data/fim_performance/fim_4_5_2_11/fim_performance_catchments.gpkg -b all

and the output mounts were against efs for "data", and "outputs" was against the local prod2 EC2 instead of the efs mount for "outputs", but that is likely not relavent.

CarsonPruitt-NOAA commented 2 weeks ago

It appears that there are only NWS test cases in /data/fim_performance/fim_4_5_2_11/fim_performance_catchments.gpkg despite you specifying to run "all" in your command. I'm doing some additional tests with the code to see what may have happened.

CarsonPruitt-NOAA commented 2 weeks ago

I ran a smaller test to include only one HUC per benchmark type and the results look as expected. The reprojection step takes a really long time, so I'm going to try a full run with that step removed. We can always reproject it afterwards with OGR.

RobHanna-NOAA commented 2 weeks ago

yes. The full run takes 12 to 14 hours generally.

RobHanna-NOAA commented 2 weeks ago

I always copy / paste my comamnds when I run them, but It is worth me trying again. Maybe I didnt' copy the 'all' command?

CarsonPruitt-NOAA commented 2 weeks ago

-b is a required argument, so it wouldn't have run without putting something there. Is there a possibility that the /previous_fim/fim_4_5_2_11 directory was in the process of being synced from on-prem servers? Or were the alpha test results not fully synced from on-prem? Since test_case_by_hydroid.py requires those alpha test results in /data/test_cases/.../official_versions/... an incomplete dataset could have tripped it up.

RobHanna-NOAA commented 2 weeks ago

From what I can tell.. all of the 4.5.2.11 test case data is there and no wholes I can see by spot checking it.