search

isce-framework / isce2-docs

Documents and tutorials for isce2
133 stars 86 forks source link

TopsApp with Conda Install and Apparent Naming Issues #11

Closed cmarshak closed 3 years ago

cmarshak commented 3 years ago

I have installed isce2 using conda (today) on my mac using:

conda create --name isce2_2020 python=3.6
conda activate isce2_2020
conda install -c conda-forge isce2
conda install -c conda-forge jupyter
python -m ipykernel install --user --name isce2_2020

Activating the above environment and then running this notebook using the above kernel, I add the following cell to update the path before running terminal command lines:

import site
from pathlib import Path

isce_application_path = Path(site.getsitepackages()[0] + '/isce/applications/')
os.environ['PATH'] += (':' + str(isce_application_path))
isce_application_path.exists(), isce_application_path

The latter points to the correct applications path that was just installed and indeed !which topsApp.py shows the same location. However, I get the following error after running !topsApp.py --dostep=startup.

This is the Open Source version of ISCE.
Some of the workflows depend on a separate licensed package.
To obtain the licensed package, please make a request for ISCE
through the website: https://download.jpl.nasa.gov/ops/request/index.cfm.
Alternatively, if you are a member, or can become a member of WinSAR
you may be able to obtain access to a version of the licensed sofware at
https://winsar.unavco.org/software/isce
Error. The attribute corresponding to the key "reference" is not present in the object "<class '__main__.TopsInSAR'>".
Possible causes are the definition in the xml file of such attribute that is no longer defined 
in the object "<class '__main__.TopsInSAR'>" or a spelling error

Thank you for your help.

Related

I have tried to run the reference notebook on Open Sar Lab: cloning the repository and then running through it. It seems to work there.

!topsApp.py --dostep=preprocess produces the following output:

Here is the output after this cell is run:

This is the Open Source version of ISCE.
Some of the workflows depend on a separate licensed package.
To obtain the licensed package, please make a request for ISCE
through the website: https://download.jpl.nasa.gov/ops/request/index.cfm.
Alternatively, if you are a member, or can become a member of WinSAR
you may be able to obtain access to a version of the licensed sofware at
https://winsar.unavco.org/software/isce
2020-11-11 19:12:50,805 - isce.insar - INFO - ISCE VERSION = 2.3, RELEASE_SVN_REVISION = 2531,RELEASE_DATE = 20190112, CURRENT_SVN_REVISION = 2531
ISCE VERSION = 2.3, RELEASE_SVN_REVISION = 2531,RELEASE_DATE = 20190112, CURRENT_SVN_REVISION = 2531
Step processing
Running step preprocess
Input XML files:  ['/vsizip/../data/slcs/S1A_IW_SLC__1SDV_20200511T135117_20200511T135144_032518_03C421_7768.zip/S1A_IW_SLC__1SDV_20200511T135117_20200511T135144_032518_03C421_7768.SAFE/annotation/s1a-iw3-slc-vv-20200511t135118-20200511t135143-032518-03c421-006.xml']
Input TIFF files:  ['/vsizip/../data/slcs/S1A_IW_SLC__1SDV_20200511T135117_20200511T135144_032518_03C421_7768.zip/S1A_IW_SLC__1SDV_20200511T135117_20200511T135144_032518_03C421_7768.SAFE/measurement/s1a-iw3-slc-vv-20200511t135118-20200511t135143-032518-03c421-006.tiff']
Manifest files:  ['/vsizip/../data/slcs/S1A_IW_SLC__1SDV_20200511T135117_20200511T135144_032518_03C421_7768.zip/S1A_IW_SLC__1SDV_20200511T135117_20200511T135144_032518_03C421_7768.SAFE/manifest.safe']
MANS:  ../data/slcs/S1A_IW_SLC__1SDV_20200511T135117_20200511T135144_032518_03C421_7768.zip S1A_IW_SLC__1SDV_20200511T135117_20200511T135144_032518_03C421_7768.SAFE/manifest.safe
Setting IPF version to :  003.20
Extracting orbit from Orbit File:  ../data/orbits/S1A_OPER_AUX_POEORB_OPOD_20200531T120757_V20200510T225942_20200512T005942.EOF
Number of Bursts before cropping:  9
Number of Bursts after cropping:  4
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Updating burst number from 4 to 1
Updating burst number from 5 to 2
Updating burst number from 6 to 3
Updating burst number from 7 to 4
Input XML files:  ['/vsizip/../data/slcs/S1B_IW_SLC__1SDV_20200517T135026_20200517T135056_021622_0290CB_99E2.zip/S1B_IW_SLC__1SDV_20200517T135026_20200517T135056_021622_0290CB_99E2.SAFE/annotation/s1b-iw3-slc-vv-20200517t135028-20200517t135056-021622-0290cb-006.xml']
Input TIFF files:  ['/vsizip/../data/slcs/S1B_IW_SLC__1SDV_20200517T135026_20200517T135056_021622_0290CB_99E2.zip/S1B_IW_SLC__1SDV_20200517T135026_20200517T135056_021622_0290CB_99E2.SAFE/measurement/s1b-iw3-slc-vv-20200517t135028-20200517t135056-021622-0290cb-006.tiff']
Manifest files:  ['/vsizip/../data/slcs/S1B_IW_SLC__1SDV_20200517T135026_20200517T135056_021622_0290CB_99E2.zip/S1B_IW_SLC__1SDV_20200517T135026_20200517T135056_021622_0290CB_99E2.SAFE/manifest.safe']
MANS:  ../data/slcs/S1B_IW_SLC__1SDV_20200517T135026_20200517T135056_021622_0290CB_99E2.zip S1B_IW_SLC__1SDV_20200517T135026_20200517T135056_021622_0290CB_99E2.SAFE/manifest.safe
Setting IPF version to :  003.20
Extracting orbit from Orbit File:  ../data/orbits/S1B_OPER_AUX_POEORB_OPOD_20200606T110735_V20200516T225942_20200518T005942.EOF
Number of Bursts before cropping:  10
Number of Bursts after cropping:  4
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Warning 1: Recode from CP437 to UTF-8 failed with the error: "Invalid argument".
Updating burst number from 7 to 1
Updating burst number from 8 to 2
Updating burst number from 9 to 3
Updating burst number from 10 to 4
2020-11-11 19:13:11,040 - isce.topsinsar.runPreprocessor - INFO - 
####################################################################################################
    runPreprocessor
----------------------------------------------------------------------------------------------------
common.Input list of swaths to process:  = [3]
reference.burstLength_3 = 1515
reference.burstWidth_3 = 24492
reference.numberOfBursts_3 = 4
reference.sensor.ascendingnodetime = 2020-05-11 13:12:30.117289
reference.sensor.family = topsswathslc
reference.sensor.folder = None
reference.sensor.imagingmode = TOPS
reference.sensor.mission = S1A
reference.sensor.name = topsswathslc_name
reference.sensor.numberofbursts = 4
reference.sensor.processingfacility = DLR-Oberpfaffenhofen, Germany
reference.sensor.processingsoftwareversion = 003.20
reference.sensor.processingsystem = Sentinel-1 IPF
reference.sensor.spacecraftname = Sentinel-1
secondary.burstLength_3 = 1513
secondary.burstWidth_3 = 24158
secondary.numberOfBursts_3 = 4
secondary.sensor.ascendingnodetime = 2020-05-17 13:11:48.571324
secondary.sensor.family = topsswathslc
secondary.sensor.folder = None
secondary.sensor.imagingmode = TOPS
secondary.sensor.mission = S1B
secondary.sensor.name = topsswathslc_name
secondary.sensor.numberofbursts = 4
secondary.sensor.processingfacility = DLR-Oberpfaffenhofen, Germany
secondary.sensor.processingsoftwareversion = 003.20
secondary.sensor.processingsystem = Sentinel-1 IPF
secondary.sensor.spacecraftname = Sentinel-1
####################################################################################################
Dumping the application's pickle object _insar to file  PICKLE/preprocess
The remaining steps are (in order):  ['computeBaselines', 'verifyDEM', 'topo', 'subsetoverlaps', 'coarseoffsets', 'coarseresamp', 'overlapifg', 'prepesd', 'esd', 'rangecoreg', 'fineoffsets', 'fineresamp', 'ion', 'burstifg', 'mergebursts', 'filter', 'unwrap', 'unwrap2stage', 'geocode', 'denseoffsets', 'filteroffsets', 'geocodeoffsets']

General Question

Can we use conda isce2 with these notebooks? Thank you!

cmarshak commented 3 years ago

So I guess it has to do with the renaming discussed here: https://github.com/isce-framework/isce2/releases/tag/v2.4.0.

I guess I am curious for end-users will the conda eventually reflect 2.4?

rtburns-jpl commented 3 years ago

Sorry, I think the reported version is a bit behind. Our conda package is currently on version 2.4.1 so if you installed recently you should be current (despite the 2.3 printout). I wasn't able to find any usage of the old nomenclature in the notebooks you're using, so I'm not sure on the cause.

cmarshak commented 3 years ago

Thank you, Ryan. I installed via conda 5 days ago when I first posted the comment using the above.

conda create --name isce2_2020 python=3.6
conda activate isce2_2020
conda install -c conda-forge isce2
conda install jupyter
conda install -c conda-forge notebook
python -m ipykernel install --user --name isce2_2020

FYI, I cloned the exact same notebook into opensarlab and it ran all the way through.

For reference, when I activate the enviornment from above, here is my terminal with version verification.


(isce2_2020) --- docker-volumes/TOPS » ipython
Python 3.6.11 | packaged by conda-forge | (default, Aug  5 2020, 20:19:23)
Type "copyright", "credits" or "license" for more information.

IPython 5.8.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload
Warning: disable autoreload in ipython_config.py to improve performance.

In [1]: import isce
This is the Open Source version of ISCE.
Some of the workflows depend on a separate licensed package.
To obtain the licensed package, please make a request for ISCE
through the website: https://download.jpl.nasa.gov/ops/request/index.cfm.
Alternatively, if you are a member, or can become a member of WinSAR
you may be able to obtain access to a version of the licensed sofware at
https://winsar.unavco.org/software/isce

In [2]: isce.__version__
Out[2]: '2.3'
cmarshak commented 3 years ago

Piyush posted this on the ISCE2 forum:

Conda is up to date. This is very likely due to not updating conda for a while on the user side or pinning some other requirement to a very old version
Most likely candidate - for some reason you are still using python 3.6

I will retry his suggestions and repost. Note: I explicitly used python 3.6.

cmarshak commented 3 years ago

I reinstalled python with the newest anaconda distribution. After that, I did the following:

conda update -n base conda
conda create --name isce2_2020 python=3.8
conda activate isce2_2020
conda install -c conda-forge isce2
conda install -c conda-forge jupyter
python -m ipykernel install --user --name isce2_2020

I am not sure if it was the python version, anaconda channels I was using, an old version of conda or some combination. Nonetheless, the referenced notebook does not crash at the beginning as it was previously.

In the interactive terminal, it still says

>>> import isce
This is the Open Source version of ISCE.
Some of the workflows depend on a separate licensed package.
To obtain the licensed package, please make a request for ISCE
through the website: https://download.jpl.nasa.gov/ops/request/index.cfm.
Alternatively, if you are a member, or can become a member of WinSAR
you may be able to obtain access to a version of the licensed sofware at
https://winsar.unavco.org/software/isce
>>> isce.__version__
'2.3'

But I take that is not relevant?

I am closing this and hope this will provide help for future end-users. Thank you.

rtburns-jpl commented 3 years ago

Great! Glad that fixed it. I agree that the old version is really confusing, I think that it was neglected when we switched from SVN to git. I updated the release history file so the next release will show the correct version info.