AComp on a Landsat-8 image not working

[egiron]

GBDXTools New Issue

Please provide some background information:

GBDX Username

[e dot egiron dot e at gmail dot com]

GBDX Account Tier

• [ ] Community (Free)
• [ ] Notebooks Only
• [ ] GBDX Starter
• [ x] GBDX Enterprise
• [ ] [other, please clarify]

GBDXTools Version

[ 0.16.5 ]

Python Version

• [ x ] 2
• [ x ] 3

Environment

• [ x ] GBDX Notebooks
• [ x ] Local

Name: gbdxtools Version: 0.16.5 Location: /anaconda3/envs/gbdx_env27/lib/python2.7/site-packages

Description

Running simple task using the Example at AComp

Expected behavior:

LandSat8 images corrected or not but saved it to the S3 gbd-customer-data path

Actual behavior:

it produces the following error trying to process the landsat image :

'stderr': u'Traceback (most recent call last):\n File "/dg/aws/apps/AOP-Processing/aws/gbdx/aop_strip_processor_task/aop_strip_processor_task_runner.py", line 661, in <module>\n task.invoke()\n File "/dg/aws/apps/AOP-Processing/aws/gbdx/aop_strip_processor_task/aop_strip_processor_task_runner.py", line 162, in invoke\n raise AopError("No SOLIs found in input directory")\n__main__.AopError: **No SOLIs found in input directory**\n', 'taskType': u'AOP_Strip_Processor:0.0.4'},

it works well with WorldView 3 imagery but fail with LandSat 8 (eg. LC80340412017095LGN00)

Example

# Run Atmospheric Compensation (AComp) on Landsat-8 data
    from gbdxtools import Interface
    gbdx = Interface()

    # The data input and lines must be edited to point to an authorized customer S3 location)
    acomp = gbdx.Task('AComp', data='s3://gbd-customer-data/CustomerAccount#/PathToImage/')
    workflow = gbdx.Workflow([acomp])
    # Edit the following line(s) to reflect specific folder(s) for the output file (example location provided)
    workflow.savedata(acomp.outputs.data, location='S3 gbd-customer-data location/<customer account>/output directory')
    workflow.execute()

    print workflow.id
    print workflow.status

Also using: AOP_Strip_Processor

aoptask = gbdx.Task("AOP_Strip_Processor", 
                    data='s3://landsat-pds/c1/L8/034/041/LC08_L1TP_034041_20170405_20170414_01_T1', 
                    enable_acomp=False, enable_pansharpen=False)

it fails too.

[drwelby]

Can you try this approach:

gbdx = Interface()

# The data input and lines must be edited to point to an authorized customer S3 location)
acomp = gbdx.Task('AComp:1.1.32', data='s3://landsat-pds/c1/L8/139/045/LC08_L1TP_139045_20170304_20170316_01_T1/')
workflow = gbdx.Workflow([acomp])
# Edit the following line(s) to reflect specific folder(s) for the output file (example location provided)
workflow.savedata(acomp.outputs.data, location='landsat_test')
workflow.execute()

print workflow.id 
print workflow.status

Some notes:

• AComp tasks need to have a version, like AComp:1.1.32
• The file path should be the path to the directory of Landsat images. That directory must contain a *_MLT.txt file.
• The acomped files will be saved in a directory in your customer S3 bucket as specified in location='landsat_test'

[egiron]

Thanks a lot for your quick response and It really worked!!!

When using only "Acomp", seems it takes the version "1.2.22" and not the last version "1.2.4".

gbdx.task_registry.get_definition('AComp')

Perhaps anyone would hope the last versions will work similar to "1.1.32" and better.

But after trying your recommended approach, it works very well.

Thanks again!!!

[drwelby]

Glad to hear it is working well for you.

When using only "Acomp", seems it takes the version "1.2.22" and not the last version "1.2.4".

That should be correct, the list is sorted lexically, so patch version 4 is placed after version 22. This causes temporary confusion for everyone...

[heidimhurst]

I encountered this while trying to troubleshoot using AOI_Strip_Processor after running a crop task.

@drwelby can you explain what the _MLT.txt file should look like? If that's something that should be auto-generated or included, do you know how I could generate it? It doesn't seem to be an output of the gdalcrop module, for example.

[drwelby]

@heidimhurst I don't know much about the internals of that task, just that the *_MLT.txt file is Landsat metadata that's included in the S3 buckets.

[heidimhurst]

np, thanks @drwelby :)