davidfrantz / force

Framework for Operational Radiometric Correction for Environmental monitoring
GNU General Public License v3.0
172 stars 50 forks source link

Error when using force-level1-landsat search with --forcelogs #264

Closed JariPekko closed 1 year ago

JariPekko commented 1 year ago

Hi! I have an AOI with older level-2 Landsat data which i want to update. I'm trying to download the newest Landsat data with force-level1-landsat search and the --forcelogs option to only download data i haven't processed yet. I tried the options --no-action and --download and both times the same error appeared. I double checked the log directory. The process is aborted due to the error.

Command and Error Message

dforce force-level1-landsat search /landsat/Landsat_tiles.txt /landsat/images/ --cloudcover 0,70 --queue-file /landsat/queue.txt --secret /landsat/usgs_m2m_access.txt --forcelogs /report/l2_log/ --download

Sensor(s): TM, ETM, OLI
Tile(s): 170071,170072,170073,171071,171072,171073,171074,172069,172070,172071,172072,172073,172074,172075,173068,173069,173070,173071,173072,173073,173074,173075,174069,174070,174071,174072,174073,174074,174075,175070,175071,175072,175073,175074,175075,176070,176071,176072,176073,176074,176075,177070,177071,177072,177073,177074,177075,178070,178071,178072,178073,178074,178075,179070,179072,179073,179074,179075,180072,180073
Date range: 1970-01-01 to 2023-01-16
Included months: 1,2,3,4,5,6,7,8,9,10,11,12
Cloud cover: 0% to 70%

Warning: The M2M API only allows requesting 15000 scenes/15 min. force-level1-landsat will pause for 15 mins if rate limiting occurs.
39190 Landsat Level 1 scenes matching criteria found
20.81 TB data volume found

Checking file system for FORCE Level-2 processing log files.
Traceback (most recent call last):
  File "/home/docker/bin/force-level1-landsat", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/landsatlinks/cli.py", line 145, in main
    product_ids_logs = utils.find_files(search_path=log_path, search_type='log', recursive=True)
  File "/usr/local/lib/python3.8/dist-packages/landsatlinks/utils.py", line 63, in find_files
    if filename not in aria_tempfiles:
UnboundLocalError: local variable 'aria_tempfiles' referenced before assignment

Setup FORCE version 3.7.10 using Docker Ubuntu 20.04.5 LTS Linux Server 500G RAM, 80 CPUs

ernstste commented 1 year ago

Hi Jari,

try pulling the latest image, this should be fixed.

Stefan

JariPekko commented 1 year ago

Hi Stefan,

I pulled the latest image and it works now, thank you!