pip installed gips_process not working in venv on rio

[ircwaves]

I installed gips into a virtual environment on rio. gips_inventory Landsat is working, but running gips_process Landsat bags out complaining that I need to install pysolar (transcript at bottom).

I'm sure that pysolar is installed:

(riogips)icooke@rio:~/.virtualenvs/riogips/lib/python2.7/site-packages/gips$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysolar
>>> 
(riogips)icooke@rio:~/.virtualenvs/riogips/lib/python2.7/site-packages/gips$ \
    ls ../pysolar
constants.py   elevation.pyc  radiation.py   simulate.pyc  time.py   util.pyc
constants.pyc  __init__.py    radiation.pyc  solar.py      time.pyc
elevation.py   __init__.pyc   simulate.py    solar.pyc     util.py

Thoughts?

(riogips)icooke@rio:~/.virtualenvs/riogips/lib/python2.7/site-packages/gips$ \
    gips_process Landsat -t 021032 -d 2015-01-01,2015-1-16 -v5 -p ndvi acca lswi ndti crc
GIPS Data Processing (v0.8.0rc3)
Retrieving inventory for site tiles
Processing [acca crc lswi ndti ndvi] on 1 dates (1 files)
LC80210322015015LGN00_B1: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B2: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B3: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B4: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B5: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B6: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B7: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B9: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B10: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B11: GeoResource Open (use_count = 1)
LC80210322015015LGN00_B1: read in 0:00:14.116555
Running atmospheric model (6S)
Traceback (most recent call last):
  File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/data/landsat.py", line 303, in process
    atm6s = SIXS(visbands, wvlens, geo, self.metadata['datetime'], sensor=self.sensor_set[0])
  File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/atmosphere.py", line 98, in __init__
    geometry['zenith'], geometry['azimuth'])
  File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/Py6S/Params/geometry.py", line 66, in from_time_and_location
    raise ImportError("To set the geometry from a time and location you must have the PySolar module installed.\nTo install this, run 'pip install pysolar' at the command line.")
ImportError: To set the geometry from a time and location you must have the PySolar module installed.
To install this, run 'pip install pysolar' at the command line.

Traceback (most recent call last):
  File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/inventory.py", line 257, in process
    self.data[date].process(*args, **kwargs)
  File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/tiles.py", line 77, in process
    [t.process(*args, products=self.products.products, **kwargs) for t in self.tiles.values()]
  File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/data/landsat.py", line 308, in process
    raise Exception('Problem running 6S atmospheric model: %s' % e)
Exception: Problem running 6S atmospheric model: To set the geometry from a time and location you must have the PySolar module installed.
To install this, run 'pip install pysolar' at the command line.

[bhbraswell]

It looks like the ImportError is raised manually by 6S, so that’s consistent with pysolar actually being installed.

On Jun 3, 2015, at 4:12 PM, ircwaves notifications@github.com wrote:

I installed gips into a virtual environment on rio. gips_inventory Landsat is working, but running gips_process Landsat bags out complaining that I need to install pysolar (transcript at bottom).

I'm sure that pysolar is installed:

(riogips)icooke@rio:~/.virtualenvs/riogips/lib/python2.7/site-packages/gips$ python Python 2.7.6 (default, Mar 22 2014, 22:59:56) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information.

import pysolar

(riogips)icooke@rio:~/.virtualenvs/riogips/lib/python2.7/site-packages/gips$ \ ls ../pysolar constants.py elevation.pyc radiation.py simulate.pyc time.py util.pyc constants.pyc init.py radiation.pyc solar.py time.pyc elevation.py init.pyc simulate.py solar.pyc util.py Thoughts?

(riogips)icooke@rio:~/.virtualenvs/riogips/lib/python2.7/site-packages/gips$ \ gips_process Landsat -t 021032 -d 2015-01-01,2015-1-16 -v5 -p ndvi acca lswi ndti crc GIPS Data Processing (v0.8.0rc3) Retrieving inventory for site tiles Processing [acca crc lswi ndti ndvi] on 1 dates (1 files) LC80210322015015LGN00_B1: GeoResource Open (use_count = 1) LC80210322015015LGN00_B2: GeoResource Open (use_count = 1) LC80210322015015LGN00_B3: GeoResource Open (use_count = 1) LC80210322015015LGN00_B4: GeoResource Open (use_count = 1) LC80210322015015LGN00_B5: GeoResource Open (use_count = 1) LC80210322015015LGN00_B6: GeoResource Open (use_count = 1) LC80210322015015LGN00_B7: GeoResource Open (use_count = 1) LC80210322015015LGN00_B9: GeoResource Open (use_count = 1) LC80210322015015LGN00_B10: GeoResource Open (use_count = 1) LC80210322015015LGN00_B11: GeoResource Open (use_count = 1) LC80210322015015LGN00_B1: read in 0:00:14.116555 Running atmospheric model (6S) Traceback (most recent call last): File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/data/landsat.py", line 303, in process atm6s = SIXS(visbands, wvlens, geo, self.metadata['datetime'], sensor=self.sensor_set[0]) File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/atmosphere.py", line 98, in init geometry['zenith'], geometry['azimuth']) File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/Py6S/Params/geometry.py", line 66, in from_time_and_location raise ImportError("To set the geometry from a time and location you must have the PySolar module installed.\nTo install this, run 'pip install pysolar' at the command line.") ImportError: To set the geometry from a time and location you must have the PySolar module installed. To install this, run 'pip install pysolar' at the command line.

Traceback (most recent call last): File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/inventory.py", line 257, in process self.data[date].process(_args, _kwargs) File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/tiles.py", line 77, in process [t.process(_args, products=self.products.products, _kwargs) for t in self.tiles.values()] File "/home/icooke/.virtualenvs/riogips/local/lib/python2.7/site-packages/gips/data/landsat.py", line 308, in process raise Exception('Problem running 6S atmospheric model: %s' % e) Exception: Problem running 6S atmospheric model: To set the geometry from a time and location you must have the PySolar module installed. To install this, run 'pip install pysolar' at the command line. — Reply to this email directly or view it on GitHub https://github.com/gipit/gips/issues/39.

[ircwaves]

Oh sweet. Looks like Pysolar version 0.7, used to be pysolar at version 0.5.

[ircwaves]

And with those tweaks, and actually filling out the settings.py AOD dictionary. This is now working. Thanks @bhbraswell !

[bhbraswell]

I’m not sure where to fix this. Change the name of the repo to lower case or (my preference) use lower() on the repository name where it is used.

rbraswell@rio:~$ gips_process landsat -p ndvi -t 013028 -d 2014-236 -v5 GIPS Data Processing (v0.8.0) Retrieving inventory for site tiles Processing [ndvi] on 1 dates (1 files) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/gips-0.8.0-py2.7.egg/gips/data/landsat/landsat.py", line 294, in process img = self._readraw() File "/usr/local/lib/python2.7/dist-packages/gips-0.8.0-py2.7.egg/gips/data/landsat/landsat.py", line 611, in _readraw if settings().REPOS[self.Repository.name]['extract']: KeyError: 'Landsat'

[ircwaves]

tolower would work. Based on checking gips.data.core.Repository, it looks like the don't-repeat-yourself thing to do would be to use inherited class method get_settings(cls, key) along the lines of:

if self.get_setting('extract'):

This is probably a bug that should be fixed everywhere that the repos settings are accessed.

[matthewhanson]

The name shouldn't be Landsat I don't think. I had changed everything to be lower case versions. Where is the name Landsat coming from? You could change it to use tolower but there might be additional problems elsewhere On Sep 15, 2015 11:38 PM, "ircwaves" notifications@github.com wrote:

Reopened #39 https://github.com/gipit/gips/issues/39.

— Reply to this email directly or view it on GitHub https://github.com/gipit/gips/issues/39#event-409865380.

[ircwaves]

looks like the text string landsatRepository.name was not updated.

Given the gips.data.core.Repository.get_setting method, I think that it is OK for the .name string to be anything...unless you just want to have it consistent across all classes, then perhaps to keep it DRY the move would be to make it a property of the repository class that drops 'Repository' from the class name and returns that?

[bhbraswell]

agree - e.g. Modis is still “Modis” and Daymet is “Daymet”.

On Sep 15, 2015, at 11:50 AM, ircwaves notifications@github.com wrote:

looks like the text string landsatRepository.name was not updated.

Given the gips.data.core.Repository.get_setting method, I think that it is OK for the .name string to be anything...unless you just want to have it consistent across all classes, then perhaps to keep it DRY the move would be to make it a property of the repository class that drops 'Repository' from the class name and returns that?

— Reply to this email directly or view it on GitHub https://github.com/gipit/gips/issues/39#issuecomment-140438632.

[ircwaves]

I believe this is fixed.