scons issue with python 2 and 3

willkochtitzky commented 4 years ago

Hi ISCE team,

First off, thanks for a developing this neat package. I look forward to using it.

When I go to install isce2 using scons this is what I get:

Will-Kochtitzkys-Mac:isce2 willkochtitzky$ scons install
scons: Reading SConscript files ...
Building with scons from python2
Checking for C header file Python.h... yes
Checking for C header file fftw3.h... yes
Checking for C header file hdf5.h... yes
Checking for C header file X11/Xlib.h... yes
Checking for C header file Xm/Xm.h... yes
Checking for C header file omp.h... yes
Checking for C library hdf5... yes
Checking for C library fftw3f... yes
Checking for C library Xm... yes
Checking for C library Xt... yes
Checking for F include fftw3 ... yes
GDAL version: 3.0.3

Checking for C++ header file gdal_priv.h... yes
Checking for C library gdal... yes
Scons appears to find everything needed for installation
Checking whether cython3 program exists.../Users/willkochtitzky/miniconda3/bin/cython3
User did not request CUDA support. Add ENABLE_CUDA = True to SConfigISCE to enable CUDA support
TypeError: makedirs() got an unexpected keyword argument 'exist_ok':
  File "/Users/willkochtitzky/bin/ISCE_SRC/isce2/SConstruct", line 219:
    os.makedirs(inst, exist_ok=True)

It seems like I am having a scons versioning issue with python 2 and 3. I have attached my config.log file as well.

I don't understand what is happening here but when I enter: /usr/bin/env python3 $(which scons)

This is what I get:

(base) Will-Kochtitzkys-Mac:isce2 willkochtitzky$ /usr/bin/env python3 $(which scons)
scons: Reading SConscript files ...
Building with scons from python3
Checking for C header file Python.h... yes
Checking for C header file fftw3.h... yes
Checking for C header file hdf5.h... yes
Checking for C header file X11/Xlib.h... yes
Checking for C header file Xm/Xm.h... yes
Checking for C header file omp.h... yes
Checking for C library hdf5... yes
Checking for C library fftw3f... yes
Checking for C library Xm... yes
Checking for C library Xt... yes
Checking for F include fftw3 ... yes
GDAL version: 3.0.3

Checking for C++ header file gdal_priv.h... yes
Checking for C library gdal... yes
Scons appears to find everything needed for installation
Checking whether cython3 program exists.../Users/willkochtitzky/miniconda3/bin/cython3
User did not request CUDA support. Add ENABLE_CUDA = True to SConfigISCE to enable CUDA support
Building with scons from python2
Checking for F include fftw3 ... yes
GDAL version: 3.0.3

Scons appears to find everything needed for installation
User did not request CUDA support. Add ENABLE_CUDA = True to SConfigISCE to enable CUDA support
TypeError: makedirs() got an unexpected keyword argument 'exist_ok':
  File "/Users/willkochtitzky/bin/ISCE_SRC/isce2/SConstruct", line 219:
    os.makedirs(inst, exist_ok=True)
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.

This seems like an improvement that I am initially running in python 3 but then I get kicked back to python 2.

I have no clue how to fix this, but would be very appreciate of any advice!

Thanks! Will Kochtitzky University of Ottawa

config.log

rtburns-jpl commented 4 years ago

I added a line which isn't python2-compatible to the scons config. Is it still common to use python2 to invoke scons? If so I will revert that change.

rtburns-jpl commented 4 years ago

As for your second error, I believe our scons script invokes itself recursively at some point but may not use the same interpreter version it was originally called with. I think we should be able to fix that as well.

willkochtitzky commented 4 years ago

Hi Ryan, thanks for your rapid response on this. I have no clue about scons and python 2 and 3 compatibility. Do you have tips for how I could run scons in python 3? It sounds like either of these solutions are potentially good. I am not sure how to do them, but appreciate your help with this. I will await action on your end. Thanks!

rtburns-jpl commented 4 years ago

It looks like what you did to invoke scons with python 3 worked, but the recursive call here has the default executable hardcoded, so you'd need to change that as well: https://github.com/isce-framework/isce2/blob/435e802f04685b0eb43b1e3d44a9850631cefd7b/SConstruct#L226

And here's the line that's causing the error - you could also just comment this out as a temporary workaround: https://github.com/isce-framework/isce2/blob/435e802f04685b0eb43b1e3d44a9850631cefd7b/SConstruct#L219

I'll wait for some more input from one of our isce2 gurus before I push a bugfix, but hopefully this gets you up and running :)

piyushrpt commented 4 years ago

I think the simplest approach might be to make SConstruct py2/py3 compatible since Macports still hasn't added a python3 variant for scons. We may want to revert to old style only in this one file:

if not os.path.exists(inst):
    os.makedirs(inst)

Still not a big fan of recursively calling scons. Dont have a simple workaround yet.

willkochtitzky commented 4 years ago

Thanks for this response. I changed the hardcoded scons -Q install to /usr/bin/env python3 $(which scons) -Q --isrerun=yes But I still got this error:

(base) Will-Kochtitzkys-Mac:isce2 willkochtitzky$ /usr/bin/env python3 $(which scons)
scons: Reading SConscript files ...
Building with scons from python3
Checking for C header file Python.h... yes
Checking for C header file fftw3.h... yes
Checking for C header file hdf5.h... yes
Checking for C header file X11/Xlib.h... yes
Checking for C header file Xm/Xm.h... yes
Checking for C header file omp.h... yes
Checking for C library hdf5... yes
Checking for C library fftw3f... yes
Checking for C library Xm... yes
Checking for C library Xt... yes
Checking for F include fftw3 ... yes
GDAL version: 3.0.3

Checking for C++ header file gdal_priv.h... yes
Checking for C library gdal... yes
Scons appears to find everything needed for installation
Checking whether cython3 program exists.../Users/willkochtitzky/miniconda3/bin/cython3
User did not request CUDA support. Add ENABLE_CUDA = True to SConfigISCE to enable CUDA support
Building with scons from python3
Checking for F include fftw3 ... yes
GDAL version: 3.0.3

Scons appears to find everything needed for installation
User did not request CUDA support. Add ENABLE_CUDA = True to SConfigISCE to enable CUDA support
No module named 'iscesys'
No module named 'iscesys'
No module named 'isceobj'
module '.' has no attribute 'getFactoriesInfo'
cython3 found.
cython3 found.
scons: `.' is up to date.
Checking for C++ header file hdf5.h... yes
Checking for C library hdf5... yes
No module named 'iscesys'
No module named 'iscesys'
No module named 'isceobj'
module '.' has no attribute 'getFactoriesInfo'
cython3 found.
cython3 found.
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.

Is this related or something different? From my reading I didn't see someone else with a similar error in the forum.

Thanks!

EJFielding commented 4 years ago

I am having the same problem with scons failing due to the python2 incompatible change in the main isce2/SConstruct. I tried the fix that @piyushrpt suggested in that file, but then it found the same incompatible use of exist_ok in isce2/configuration/buildHelper.py. How many places was the code changed to the Python3 syntax?

EJFielding commented 4 years ago

Oh, I see in #112 that there were 115 files changed. Is there a way for me to locally revert that one change in git?

rtburns-jpl commented 4 years ago

@willkochtitzky @EJFielding Let me know if the PR I just merged fixes your issues. Fyi the change was in 23f2ffa, so you could git revert 23f2ffa to create a revert commit or git checkout 23f2ffa^ to checkout the previous commit.

EJFielding commented 4 years ago

Thanks @rtburns-jpl for the quick PR fix. The Python2 scons now runs for me.

willkochtitzky commented 4 years ago

I ended up finding this script and installed it that way: https://gist.github.com/jhkennedy/5a22251dd6f1b73db1968517f00a8af6

Thanks for fixing it though! Good to know that should work now. Thanks!

SunJ-Stars commented 4 years ago

Hello, ISCE team, The following error message appears when I install isce v2.3.3. ... Checking for C++ header file hdf5.h... (cached) yes Checking for C library hdf5... (cached) yes No module named 'iscesys' No module named 'iscesys' No module named 'isceobj' module '.' has no attribute 'getFactoriesInfo' cython3 found. cython3 found. scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets.

Where should I modify the script?Thank you.

sasssssssssa commented 3 years ago

Hey SunJ,

Try to install it using Conda.

To install Conda: https://docs.conda.io/en/latest/miniconda.html

To install ISCE: https://github.com/conda-forge/isce2-feedstock

To verify your installation: python3

import ISCE isce.version.release_version from stripmapApp import Insar

Have a good day, Sam

isce-framework / isce2

scons issue with python 2 and 3 #115