search

Earth-Information-System / fireatlas

4 stars 2 forks source link

V2 Algo Runs are Failing with no AFP Coming Back #93

Closed ranchodeluxe closed 1 month ago

ranchodeluxe commented 1 month ago

Problem

BorealNA and CONUS runs have been failing since the 27th b/c the active fire pixels coming back from FireIO.read_AFP are None. This could be a simple fix but then again with all the false positives in the data we should always have some active fire pixels

2024-07-30 15:11:48,982 - FireLog - INFO - [2024, 7, 28, 'PM']
2024-07-30 15:12:57,774 - FireLog - INFO - New fire clusters of 500 at this time step
2024-07-30 15:15:28,896 - FireLog - INFO - expanding fires 151.49194383621216
2024-07-30 15:15:44,443 - FireLog - INFO - merging fires 15.503741264343262
2024-07-30 15:15:44,443 - FireLog - INFO - fids_expand: [8553, 8806, 9008, 11219, 11407, 11579, 11581, 11636, 11638, 11745, 11982, 12319, 12610, 12625, 12712, 12718, 12770, 12906, 13127, 13363, 13387, 13388, 13395, 13419, 13469, 13493, 13691, 13728, 13819, 13856, 13928, 13965, 13996, 14018, 14090, 14123, 14167, 14168, 14211, 14219, 14259, 14284, 14317, 14321, 14347, 14438, 14451, 14464, 14569, 14585, 14588, 14620, 14637, 14750, 14752, 14765, 14809, 14855, 14862, 14869, 14875, 14891, 14894, 14900, 14905, 14910, 14944, 14990, 15025, 15056, 15063]
2024-07-30 15:15:44,443 - FireLog - INFO - fids_new: [15067, 15068, 15069, 15070, 15071, 15072, 15073, 15074, 15075, 15076, 15077, 15078, 15079, 15080, 15081, 15082, 15083, 15084, 15085, 15086, 15087, 15088, 15089, 15090, 15091, 15092, 15093, 15094, 15095, 15096, 15097, 15098]
2024-07-30 15:15:44,443 - FireLog - INFO - fids_merged: [13388, 15073, 15082, 15092]
2024-07-30 15:15:44,443 - FireLog - INFO - fids_invalid: [15083, 15087, 15088, 15093, 15098]
2024-07-30 15:16:10,640 - FireLog - INFO - 4.360967528820038 minutes used to run alg [2024, 7, 28, 'PM']
2024-07-30 15:16:10,640 - FireLog - INFO - 
2024-07-30 15:16:10,640 - FireLog - INFO - [2024, 7, 29, 'AM']
2024-07-30 15:16:10,970 - FireLog - ERROR - object of type 'NoneType' has no len()
Traceback (most recent call last):
  File "/app/fireatlas/FireRun.py", line 720, in <module>
    run_func()
  File "/app/fireatlas/FireRun.py", line 255, in BorealNA
    FireMain.Fire_Forward(tst=tst, ted=ted, restart=False, region=region)
  File "/app/fireatlas/FireMain.py", line 612, in Fire_Forward
    if len(afp) > 0:
TypeError: object of type 'NoneType' has no len()

Solution

  1. Let's get a group together at the bi-weekly meeting or after and fix this together
ranchodeluxe commented 1 month ago

the data is not available for these days and that's why everything was broken:

https://nrt4.modaps.eosdis.nasa.gov/api/v2/content/archives/FIRMS/noaa-20-viirs-c2/Global/J1_VIIRS_C2_Global_VJ114IMGTDL_NRT_2024211.txt https://nrt4.modaps.eosdis.nasa.gov/api/v2/content/archives/FIRMS/noaa-20-viirs-c2/Global/J1_VIIRS_C2_Global_VJ114IMGTDL_NRT_2024212.txt

mccabete commented 1 month ago

Ok, so to recap we found out that this was a server error of some kind. We were able to find that day with a wget statement to the firms API: https://firms.modaps.eosdis.nasa.gov/

And Shane and I looked and the data is also there for the above api in nrt3, just not nrt4. I emailed them.