Cannot concatenate 'str' and 'tuple' objects - Githubissues

lucadelu / pyModis

python library to work with MODIS data

http://www.pymodis.org/

Other

165 stars 85 forks source link

Cannot concatenate 'str' and 'tuple' objects #87

Closed Jlugol closed 7 years ago

Jlugol commented 7 years ago

Hi all ! I'm new with Python and pyModis, I successfully installed pyModis and its dependencies (no error when I import pyModis). I always get the error message "cannot concatenate 'str' and 'tuple' objects" when I execute commands as modis_download.py -r -U username -P password -p MOD09Q1.006 -t h12v04,h13v04 -f 2015-01-17 -e 2015-01-25 /home/user/desktop An hdf file, a txt file and a log file are create but they're empty except the log file which contains :

2017-05-22 15:50:26,556 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov 2017-05-22 15:50:26,739 - DEBUG - Setting read timeout to 30 2017-05-22 15:50:27,002 - DEBUG - "GET /MOLT/MOD13Q1.005 HTTP/1.1" 302 228 2017-05-22 15:50:27,029 - INFO - Starting new HTTPS connection (1): e4ftl01.cr.usgs.gov 2017-05-22 15:50:27,621 - DEBUG - Setting read timeout to 30 2017-05-22 15:50:27,800 - DEBUG - "GET /MOLT/MOD13Q1.005 HTTP/1.1" 301 253 2017-05-22 15:50:27,805 - DEBUG - Setting read timeout to 30 2017-05-22 15:50:28,016 - DEBUG - "GET /MOLT/MOD13Q1.005/ HTTP/1.1" 200 None 2017-05-22 15:50:35,806 - DEBUG - The number of days to download is: 3 2017-05-22 15:50:35,836 - DEBUG - The url is: http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2014.02.02 2017-05-22 15:50:35,837 - INFO - Starting new HTTP connection (1): e4ftl01.cr.usgs.gov 2017-05-22 15:50:36,068 - DEBUG - Setting read timeout to 30 2017-05-22 15:50:36,228 - DEBUG - "GET /MOLT/MOD13Q1.005/2014.02.02 HTTP/1.1" 302 239 2017-05-22 15:50:36,232 - INFO - Starting new HTTPS connection (1): e4ftl01.cr.usgs.gov 2017-05-22 15:50:36,720 - DEBUG - Setting read timeout to 30 2017-05-22 15:50:36,856 - DEBUG - "GET /MOLT/MOD13Q1.005/2014.02.02 HTTP/1.1" 301 264 2017-05-22 15:50:36,857 - DEBUG - Setting read timeout to 30 2017-05-22 15:50:37,814 - DEBUG - "GET /MOLT/MOD13Q1.005/2014.02.02/ HTTP/1.1" 200 None 2017-05-22 15:50:38,947 - DEBUG - The number of file to download is: 2

I'm hoping someone can help a new user like me :).

lucadelu commented 7 years ago

Hi @Sedigar, the problem should be fixed in 9aea389. You will find this bugfix in the next release otherwise update your version with the github one

Please next time report the full error messages. You will find

neteler commented 7 years ago

I also get this (just de-installed and installed pyModis with pip and the GRASS GIS addon with g.extension):

GRASS 7.2.1svn (modis_ndvi_global):~ > r.modis.download
settings=~/.rmodis product=ndvi_terra_sixteen_5600 \
>         startday=2013-01-01 endday=`date +"%Y-%m-%d"` folder=$MODIS_TMPFOLDER
WxPython missing, no GUI enabled
WARNING: Option 'tiles' not set. Downloading all available tiles
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:334:
SNIMissingWarning: An HTTPS request has been made, but the SNI
(Subject Name Indication) extension to TLS is not available on this
platform. This may cause the server to present an incorrect TLS
certificate, which can cause validation failures. You can upgrade to a
newer version of Python to solve this. For more information, see
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and may cause
certain SSL connections to fail. You can upgrade to a newer version of
Python to solve this. For more information, see
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
Downloading MODIS product <ndvi_terra_sixteen_5600>...
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and may cause
certain SSL connections to fail. You can upgrade to a newer version of
Python to solve this. For more information, see
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
Traceback (most recent call last):
  File "/home/eouser/.grass7/addons/scripts/r.modis.download", line
263, in <module>
    sys.exit(main())
  File "/home/eouser/.grass7/addons/scripts/r.modis.download", line 247, in main
    modisOgg.downloadsAllDay()
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py",
line 831, in downloadsAllDay
    self._downloadAllDaysHTTP(days)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py",
line 847, in _downloadAllDaysHTTP
    self.dayDownload(day, listFilesDown)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py",
line 809, in dayDownload
    self.downloadFile(i, file_hdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py",
line 660, in downloadFile
    self._downloadFileHTTP(filDown, filHdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py",
line 682, in _downloadFileHTTP
    "error " + sys.exc_info())
TypeError: cannot concatenate 'str' and 'tuple' objects

lucadelu commented 7 years ago

@neteler please apply the patch Friday I will release new version

neteler commented 7 years ago

Works, thanks!

neteler commented 7 years ago

... unfortunately still an issue (it shows up much later):

/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 851, in emit
Traceback (most recent call last):
  File "/home/eouser/.grass7/addons/scripts/r.modis.download", line 263, in <module>
    sys.exit(main())
  File "/home/eouser/.grass7/addons/scripts/r.modis.download", line 247, in main
    modisOgg.downloadsAllDay()
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 831, in downloadsAllDay
    self._downloadAllDaysHTTP(days)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 847, in _downloadAllDaysHTTP
    self.dayDownload(day, listFilesDown)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 809, in dayDownload
    self.downloadFile(i, file_hdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 660, in downloadFile
    self._downloadFileHTTP(filDown, filHdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 696, in _downloadFileHTTP
    self._downloadFileHTTP(filDown, filHdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 696, in _downloadFileHTTP
    self._downloadFileHTTP(filDown, filHdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 696, in _downloadFileHTTP
    self._downloadFileHTTP(filDown, filHdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 696, in _downloadFileHTTP
    self._downloadFileHTTP(filDown, filHdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 696, in _downloadFileHTTP
    self._downloadFileHTTP(filDown, filHdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 696, in _downloadFileHTTP
    self._downloadFileHTTP(filDown, filHdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 696, in _downloadFileHTTP
...
    self._downloadFileHTTP(filDown, filHdf, day)
  File "/usr/local/lib/python2.7/dist-packages/pymodis/downmodis.py", line 682, in _downloadFileHTTP
    "error {ex}".format(ex=sys.exc_info()))
  File "/usr/lib/python2.7/logging/__init__.py", line 1604, in warning
    root.warning(msg, *args, **kwargs)
  File "/usr/lib/python2.7/logging/__init__.py", line 1164, in warning
    self._log(WARNING, msg, args, **kwargs)
  File "/usr/lib/python2.7/logging/__init__.py", line 1271, in _log
    self.handle(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 1281, in handle
    self.callHandlers(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 1321, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 749, in handle
    self.emit(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 942, in emit
    StreamHandler.emit(self, record)
  File "/usr/lib/python2.7/logging/__init__.py", line 879, in emit
    self.handleError(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 802, in handleError
    None, sys.stderr)
  File "/usr/lib/python2.7/traceback.py", line 125, in print_exception
    print_tb(tb, limit, file)
  File "/usr/lib/python2.7/traceback.py", line 69, in print_tb
    line = linecache.getline(filename, lineno, f.f_globals)
  File "/usr/lib/python2.7/linecache.py", line 14, in getline
    lines = getlines(filename, module_globals)
RuntimeError: maximum recursion depth exceeded

neteler commented 7 years ago

The log file contains:

eouser@graas-node-1:~/scratch/modis/ndvi_MOD13C1.006$ cat modisMOD13C1.006.log
2017-05-23 17:38:27,697 - DEBUG - Starting new HTTPS connection (1): e4ftl01.cr.usgs.gov
2017-05-23 17:38:29,692 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006 HTTP/1.1" 301 253
2017-05-23 17:38:29,857 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006/ HTTP/1.1" 200 None
2017-05-23 17:38:30,156 - DEBUG - Starting new HTTPS connection (1): e4ftl01.cr.usgs.gov
2017-05-23 17:38:30,734 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006/2017.04.23 HTTP/1.1" 301 264
2017-05-23 17:38:30,881 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006/2017.04.23/ HTTP/1.1" 200 None
2017-05-24 11:58:57,370 - DEBUG - Starting new HTTPS connection (1): e4ftl01.cr.usgs.gov
2017-05-24 11:58:59,523 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006 HTTP/1.1" 301 253
2017-05-24 11:59:00,630 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006/ HTTP/1.1" 200 None
2017-05-24 11:59:00,930 - DEBUG - Starting new HTTPS connection (1): e4ftl01.cr.usgs.gov
2017-05-24 11:59:02,161 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006/2017.04.23 HTTP/1.1" 301 264
2017-05-24 11:59:02,318 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006/2017.04.23/ HTTP/1.1" 200 None
2017-05-24 11:59:02,322 - WARNING - Tried to downlaod with urllib but got this error (<class 'future.backports.urllib.error.URLError'>, URLError(u'unknown url type: https',), <traceback object at 0x7f7407fcc0e0>)
2017-05-24 11:59:02,323 - DEBUG - Starting new HTTPS connection (1): e4ftl01.cr.usgs.gov
2017-05-24 11:59:03,174 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006/2017.04.23/MOD13C1.A2017113.006.2017131134151.hdf.xml HTTP/1.1" 302 518
2017-05-24 11:59:03,175 - DEBUG - Starting new HTTPS connection (1): urs.earthdata.nasa.gov
2017-05-24 11:59:06,552 - DEBUG - https://urs.earthdata.nasa.gov:443 "GET /oauth/authorize?scope=uid&app_type=401&client_id=ijpRZvb9qeKCK5ctsn75Tg&response_type=code&redirect_uri=https%3A%2F%2Fe4ftl01.cr.usgs.gov%2Foauth&state=aHR0cHM6Ly9lNGZ0bDAxLmNyLnVzZ3MuZ292L01PTFQvTU9EMTNDMS4wMDYvMjAxNy4wNC4yMy9NT0QxM0MxLkEyMDE3MTEzLjAwNi4yMDE3MTMxMTM0MTUxLmhkZi54bWw HTTP/1.1" 401 None
2017-05-24 11:59:06,555 - WARNING - Tried to downlaod with requests but got this error (<type 'exceptions.KeyError'>, KeyError('content-length',), <traceback object at 0x7f7408447830>)
2017-05-24 11:59:06,555 - ERROR - Cannot download MOD13C1.A2017113.006.2017131134151.hdf.xml. Retrying...
2017-05-24 11:59:11,561 - WARNING - Tried to downlaod with urllib but got this error (<class 'future.backports.urllib.error.URLError'>, URLError(u'unknown url type: https',), <traceback object at 0x7f74084476c8>)
2017-05-24 11:59:11,562 - DEBUG - Starting new HTTPS connection (1): e4ftl01.cr.usgs.gov
2017-05-24 11:59:12,142 - DEBUG - https://e4ftl01.cr.usgs.gov:443 "GET /MOLT/MOD13C1.006/2017.04.23/MOD13C1.A2017113.006.2017131134151.hdf.xml HTTP/1.1" 302 518
2017-05-24 11:59:12,143 - DEBUG - Starting new HTTPS connection (1): urs.earthdata.nasa.gov
2017-05-24 11:59:14,096 - DEBUG - https://urs.earthdata.nasa.gov:443 "GET /oauth/authorize?scope=uid&app_type=401&client_id=ijpRZvb9qeKCK5ctsn75Tg&response_type=code&redirect_uri=https%3A%2F%2Fe4ftl01.cr.usgs.gov%2Foauth&state=aHR0cHM6Ly9lNGZ0bDAxLmNyLnVzZ3MuZ292L01PTFQvTU9EMTNDMS4wMDYvMjAxNy4wNC4yMy9NT0QxM0MxLkEyMDE3MTEzLjAwNi4yMDE3MTMxMTM0MTUxLmhkZi54bWw HTTP/1.1" 401 None
2017-05-24 11:59:14,099 - WARNING - Tried to downlaod with requests but got this error (<type 'exceptions.KeyError'>, KeyError('content-length',), <traceback object at 0x7f7407fcce18>)
...
2017-05-24 13:49:01,643 - WARNING - Tried to downlaod with requests but got this error (<type 'exceptions.RuntimeError'>, RuntimeError('maximum recursion depth exceeded in __instancecheck__',), <traceback object at 0x7f740533ce60>)
2017-05-24 13:49:01,643 - ERROR - Cannot download MOD13C1.A2017113.006.2017131134151.hdf.xml. Retrying...
2017-05-24 13:49:06,649 - WARNING - Tried to downlaod with urllib but got this error (<class 'future.backports.urllib.error.URLError'>, URLError(u'unknown url type: https',), <traceback object at 0x7f740533cef0>)
2017-05-24 13:49:06,649 - WARNING - Tried to downlaod with requests but got this error (<type 'exceptions.RuntimeError'>, RuntimeError('maximum recursion depth exceeded',), <traceback object at 0x7f74052cb170>)
2017-05-24 13:49:06,649 - ERROR - Cannot download MOD13C1.A2017113.006.2017131134151.hdf.xml. Retrying...

neteler commented 7 years ago

The "RuntimeError: maximum recursion depth exceeded" issue comes from the fact that in case of authentication error(s) it tries infinite times to re-do it... it would be better to tell the user that the authentication failed.

lucadelu commented 7 years ago

@neteler ok, so this is another problem (maybe an enhancement), I will open a new issue for this.

@Sedigar I think this issue could be closed

neteler commented 7 years ago

I have opened the "RuntimeError: maximum recursion depth exceeded" as issue #89

Jlugol commented 7 years ago

Sorry for the late answer @lucadelu, I installed python 3 instead of python 2.7 and the issue disappeared, thank you, I close the issue.