nasa-jpl / autoRIFT

A Python module of a fast and intelligent algorithm for finding the pixel displacement between two images
Apache License 2.0
216 stars 52 forks source link

Can't run autoRIFT on Sentinel-1 images #30

Closed bjmarfito closed 3 years ago

bjmarfito commented 3 years ago

I am having problems running testautoRIFT_ISCE.py using the processed Sentinel-1 images using ISCE. I've processed the images using ISCE until the mergebursts step. Then, I've run the testGeogrid.py to generate a window location which ran successfully. However, when I ran the testautoRIFT_ISCE.py an error pops up.

Below is the command I've run on the terminal using the reference and secondary slc.full files and the accompanying error message.

(isce2expe) abscissa@ABSCISSA:/mnt/f/2020_Masbate/Sentinel-1/DInSAR/autoRIFT$ python testautoRIFT_ISCE.py -m ../20200814_20200821_des_neworb/merged/reference.slc.full -s ../20200814_20200821_des_neworb/merged/secondary.slc.full -g window_location.tif
This is the Open Source version of ISCE.
Some of the workflows depend on a separate licensed package.
To obtain the licensed package, please make a request for ISCE
through the website: https://download.jpl.nasa.gov/ops/request/index.cfm.
Alternatively, if you are a member, or can become a member of WinSAR
you may be able to obtain access to a version of the licensed sofware at
https://winsar.unavco.org/software/isce
ERROR 4: ../20200814_20200821_des_neworb/merged/reference.slc.full: No such file or directory
Traceback (most recent call last):
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 465, in mmapFromISCE
    img, dataName, metaName = loadImage(fname)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 416, in loadImage
    prop,fac,misc = parser.parse(metaName)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/iscesys/Parsers/XmlParser.py", line 41, in parse
    root = ET.parse(filename)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
    tree.parse(source, parser)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/xml/etree/ElementTree.py", line 584, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '../20200814_20200821_des_neworb/merged/reference.slc.full.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 469, in mmapFromISCE
    img = loadGDALImage(fname)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 444, in loadGDALImage
    drv = ds.GetDriver()
AttributeError: 'NoneType' object has no attribute 'GetDriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "testautoRIFT_ISCE.py", line 957, in <module>
    main()
  File "testautoRIFT_ISCE.py", line 387, in main
    generateAutoriftProduct(indir_m=inps.indir_m, indir_s=inps.indir_s, grid_location=inps.grid_location,
  File "testautoRIFT_ISCE.py", line 414, in generateAutoriftProduct
    data_m = loadProduct(indir_m)
  File "testautoRIFT_ISCE.py", line 98, in loadProduct
    IMG = IML.mmapFromISCE(filename, logging)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 473, in mmapFromISCE
    raise Exception('Input file: {0} should either be an ISCE image / GDAL image. Appears to be neither'.format(fname))
Exception: Input file: ../20200814_20200821_des_neworb/merged/reference.slc.full should either be an ISCE image / GDAL image. Appears to be neither

Though, I noticed that autoRIFT is reading the xml files instead since ISCE didn't produce the .slc.full files.

I've also ran the VRT files of the slc.full files and it produced the same error. Below is the output from the terminal.

(isce2expe) abscissa@ABSCISSA:/mnt/f/2020_Masbate/Sentinel-1/DInSAR/autoRIFT$ python testautoRIFT_ISCE.py -m ../20200814_20200821_des_neworb/merged/reference.slc.full.vrt -s ../20200814_20200821_des_neworb/merged/secondary.slc.full.vrt -g window_location.tif
This is the Open Source version of ISCE.
Some of the workflows depend on a separate licensed package.
To obtain the licensed package, please make a request for ISCE
through the website: https://download.jpl.nasa.gov/ops/request/index.cfm.
Alternatively, if you are a member, or can become a member of WinSAR
you may be able to obtain access to a version of the licensed sofware at
https://winsar.unavco.org/software/isce
ERROR 4: ../20200814_20200821_des_neworb/merged/reference.slc.full.vrt: No such file or directory
Traceback (most recent call last):
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 465, in mmapFromISCE
    img, dataName, metaName = loadImage(fname)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 416, in loadImage
    prop,fac,misc = parser.parse(metaName)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/iscesys/Parsers/XmlParser.py", line 41, in parse
    root = ET.parse(filename)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
    tree.parse(source, parser)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/xml/etree/ElementTree.py", line 584, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '../20200814_20200821_des_neworb/merged/reference.slc.full.vrt.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 469, in mmapFromISCE
    img = loadGDALImage(fname)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 444, in loadGDALImage
    drv = ds.GetDriver()
AttributeError: 'NoneType' object has no attribute 'GetDriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "testautoRIFT_ISCE.py", line 957, in <module>
    main()
  File "testautoRIFT_ISCE.py", line 387, in main
    generateAutoriftProduct(indir_m=inps.indir_m, indir_s=inps.indir_s, grid_location=inps.grid_location,
  File "testautoRIFT_ISCE.py", line 414, in generateAutoriftProduct
    data_m = loadProduct(indir_m)
  File "testautoRIFT_ISCE.py", line 98, in loadProduct
    IMG = IML.mmapFromISCE(filename, logging)
  File "/home/abscissa/miniconda3/envs/isce2expe/lib/python3.8/site-packages/isce/components/isceobj/Util/ImageUtil/ImageLib.py", line 473, in mmapFromISCE
    raise Exception('Input file: {0} should either be an ISCE image / GDAL image. Appears to be neither'.format(fname))
Exception: Input file: ../20200814_20200821_des_neworb/merged/reference.slc.full.vrt should either be an ISCE image / GDAL image. Appears to be neither

If ISCE doesn't produce those files right away, what are the methods available to produce .slc.full files so I would be able to process them in autoRIFT?

bjmarfito commented 3 years ago

I've finally solved it. I've run the processing up until the dense offsets. Afterwards, the reference and secondary.slc.full files appeared on the merged folder.

My apologies for the noise.

guangbaoshen commented 2 years ago

hi, @bjmarfito , I have the same question like you , i do not know how the isce produce the .slc.full file, can you show me the comand line of how you get the .slc.full file? thank you very much.

alex-s-gardner commented 2 years ago

@jhkennedy have we seen this issue on our end?

jhkennedy commented 2 years ago

@guangbaoshen as @bjmarfito mentions, you'll need to run ISCE up to the merge bursts steps. On the command line, that'll look like:

${ISCE_HOME}/applications/topsApp.py topsApp.xml --end=mergebursts

You'll have to make the topsApp.xml file for ISCE -- I've made a python helper function to do that which might be useful to you

def format_tops_xml(reference, secondary, polarization, dem, orbits, xml_file='topsApp.xml'):
    xml_template = f"""    <?xml version="1.0" encoding="UTF-8"?>
    <topsApp>
        <component name="topsinsar">
            <component name="reference">
                <property name="orbit directory">{orbits}</property>
                <property name="auxiliary data directory">{orbits}</property>
                <property name="output directory">reference</property>
                <property name="safe">['{reference}.zip']</property>
                <property name="polarization">{polarization}</property>
            </component>
            <component name="secondary">
                <property name="orbit directory">{orbits}</property>
                <property name="auxiliary data directory">{orbits}</property>
                <property name="output directory">secondary</property>
                <property name="safe">['{secondary}.zip']</property>
                <property name="polarization">{polarization}</property>
            </component>
            <property name="demfilename">{dem}</property>
            <property name="do interferogram">False</property>
            <property name="do dense offsets">True</property>
            <property name="do ESD">False</property>
            <property name="do unwrap">False</property>
            <property name="do unwrap 2 stage">False</property>
            <property name="ampcor skip width">32</property>
            <property name="ampcor skip height">32</property>
            <property name="ampcor search window width">51</property>
            <property name="ampcor search window height">51</property>
            <property name="ampcor window width">32</property>
            <property name="ampcor window height">32</property>
        </component>
    </topsApp>
    """

    with open(xml_file, 'w') as f:
        f.write(textwrap.dedent(xml_template))
guangbaoshen commented 2 years ago

I got it, thank you very much! @jhkennedy