search

ubarsc / python-fmask

A set of command line utilities and Python modules that implement the ‘fmask’ algorithm
https://www.pythonfmask.org
GNU General Public License v3.0
75 stars 21 forks source link

some problems with python3.6? #32

Closed eamanu closed 3 years ago

eamanu commented 4 years ago

Hi,

When I try to run with python3.6 (by default) I have the next error:

$ fmask_sentinel2Stacked.py --safedir /tmp/tmpc6i7ri38/S2A_MSIL1C_20200101T142731_N0208_R053_T19JFH_20200101T174749.SAFE -o cloud.img
Traceback (most recent call last):
  File "/usr/local/bin/fmask_sentinel2Stacked.py", line 22, in <module>
    sentinel2Stacked.mainRoutine()
  File "/usr/local/lib/python3.6/dist-packages/fmask/cmdline/sentinel2Stacked.py", line 285, in mainRoutine
    resampledBands = makeStackAndAngles(cmdargs)
  File "/usr/local/lib/python3.6/dist-packages/fmask/cmdline/sentinel2Stacked.py", line 214, in makeStackAndAngles
    '-o', cmdargs.toa] + resampledBands)
  File "/usr/lib/python3.6/subprocess.py", line 306, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 287, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1295, in _execute_child
    restore_signals, start_new_session, preexec_fn)
TypeError: expected str, bytes or os.PathLike object, not NoneType

But when I python3.7 (by default) any problems:

$ fmask_sentinel2Stacked.py --safedir /tmp/tmpc6i7ri38/S2A_MSIL1C_20200101T142731_N0208_R053_T19JFH_20200101T174749.SAFE -o cloud.img
$

I don't investigate the error, but could be possible this?

neilflood commented 4 years ago

Hmmmm... I have certainly never seen this before, but I have not done detailed testing of different versions of Python. It seems more likely to be something wrong with the installation rather than in python-fmask, but I guess it is possible.

gillins commented 4 years ago

Works fine for me with Python 3.6. Not sure what to suggest... Are you using conda-forge to install?

eamanu commented 4 years ago

Hi @neilflood @gillins sorry for the delay

Nope, I am not using coda-forge. I install python-fmask (and its depedencies). And if I run fmask_usgsLandsatStacked.py on a Ubuntu 18 using py36 I have the error reported. And using fmask_usgsLandsatStacked.py on a Debian (Testing) with py37 I have:

fmask_usgsLandsatStacked.py -o cmask_Caso_C.tiff --scenedir /home/eamanu/Ascentio/dev/SCAN-TERRA_Project/CloudMask/test_cloud/test/
0...10...20...30...40...50...60...70...80...90...100 - done.

Computing Pyramid Layers...
0...10...20...30...40...50...60...70...80...90...100 - done.

Computing Statistics...
0...10...20...30...40...50...60...70...80...90...100 - done.
0...10...20...30...40...50...60...70...80...90...100 - done.
neilflood commented 4 years ago

Hi @eamanu,

Thanks for further checking.

I just did some testing using Ubuntu 18.04 and python3.6.9. I installed gdal and python3-gdal directly from apt. This seemed closest to what you were using.

I got exactly the same error as you. It seems to arise because the installs above do not install the gdal_merge.py command, which should be a part of the GDAL packages. I don't know enough about exactly how this was supposed to be packaged, but this is what causes the problem. Perhaps if I had installed things in a different order, it might have included gdal_merge.py??? I am quite unsure, but will be interested to hear if this matches your case, and whether you have any thoughts on how to avoid this.

gillins commented 4 years ago

Thanks @neilflood. Perhaps we should we be giving a better error message in this case?

neilflood commented 4 years ago

I agree about a better error message. That should be fairly straightforward.

I have been concerned about the fact that it can even happen. It appears to be a known problem with the way Ubuntu packaged things. Judging from this https://gis.stackexchange.com/questions/283528/gdal-merge-py-not-found, it would seem that one needs to have installed both the python-2 and python-3 GDAL bindings, as gdal_merge.py is packaged into python-gdal, but not into python3-gdal.

Not sure how detailed the error message should be, though. I think I prefer just to say it can't find gdal_merge.py, and suggest that they check their GDAL installation.

neilflood commented 4 years ago

@eamanu I have just pushed changes in python-fmask, to add better error messages for this particular condition. Could you please test, and let me know if you think this is sufficient. If so, we will consider doing a release with this change included.

eamanu commented 4 years ago

Hi @neilflood yes seems to be a problem on ubuntu's package. Thanks for the quick response! :) I will test it today. Thanks!!