Applied-GeoSolutions / gips

Geospatial Image Processing System
GNU General Public License v3.0
17 stars 5 forks source link

GIPS modis fetch error #477

Closed bziniti closed 6 years ago

bziniti commented 6 years ago

I can fetch the file http://e4ftl01.cr.usgs.gov/MOTA/MCD43A4.006/2016.04.01/MCD43A4.A2016092.h12v04.006.2016180010120.hdf

directly from a browser, but doesn't work from gips.

It appears my settings files as the same info (username and password for earth data login).

Help! -- thanks

bziniti@rio:~/.gips$ gips_project modis -p indices -s /scratch/bziniti/fusion_tests/fusion_test_site_modis_landsat_S2_RIforest.shp -d 2016-04-01,2016-10-01 --outdir /scratch/bziniti/fusion_tests/modis --res 30 30 -v4 --notld --fetch
GIPS Data Project (v0.8.4)
Retrieving inventory for site fusion_test_site_modis_landsat_S2_RIforest-0
Download of MCD43A4.A2016092.h12v04.006.2016180010120.hdf failed: 401 Unauthorized 
Full URL: http://e4ftl01.cr.usgs.gov/MOTA/MCD43A4.006/2016.04.01/MCD43A4.A2016092.h12v04.006.2016180010120.hdf
Download of MCD43A4.A2016093.h12v04.006.2016180011903.hdf failed: 401 Unauthorized 
Full URL: http://e4ftl01.cr.usgs.gov/MOTA/MCD43A4.006/2016.04.02/MCD43A4.A2016093.h12v04.006.2016180011903.hdf
Download of MCD43A4.A2016094.h12v04.006.2016180014003.hdf failed: 401 Unauthorized 
Full URL: http://e4ftl01.cr.usgs.gov/MOTA/MCD43A4.006/2016.04.03/MCD43A4.A2016094.h12v04.006.2016180014003.hdf
^CTraceback (most recent call last):
  File "/usr/local/bin/gips_project", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/gips/scripts/project.py", line 67, in main
    inv = DataInventory(cls, extent, t_extent, **vars(args))
  File "/usr/local/lib/python2.7/dist-packages/gips/inventory.py", line 257, in __init__
    dataclass.fetch(self.products.base, self.spatial.tiles, self.temporal, self.update)
  File "/usr/local/lib/python2.7/dist-packages/gips/data/core.py", line 807, in fetch
    cls.Asset.fetch(a, t, d)
  File "/usr/local/lib/python2.7/dist-packages/gips/data/modis/modis.py", line 201, in fetch
    listing = urllib.urlopen(mainurl).readlines()
  File "/usr/lib/python2.7/socket.py", line 519, in readlines
    line = self.readline()
  File "/usr/lib/python2.7/socket.py", line 434, in readline
    data = recv(1)
  File "/usr/lib/python2.7/ssl.py", line 750, in recv
    self._checkClosed()
KeyboardInterrupt
bziniti@rio:~/.gips$ ls
settings.py  settings.pyc  settings.py~
bziniti@rio:~/.gips$ emacs settings.py
bziniti commented 6 years ago

creation of a .netrc solved the problem. Thank you @ircwaves

bziniti@nile:~$ more .netrc
machine urs.earthdata.nasa.gov login bziniti@appliedgeosolutions.com password ************
ircwaves commented 6 years ago

And eventually we'll get the cluster upgraded and that netrc will be irrelevant (and possibly problematic ;) )