ambiguiguous metadata when of Landsat atmospheric correction when MOD08 fetch fails

[ircwaves]

based on the -v4 output below, it looks like when the fetch of the MOD08 product encounters an FTP error

error_temp: 421 Too many users are already logged on.  Please try again in 10 minutes.

it proceeds using the long term average. This gets documented in the output meta data as

AOD Source=Weighted estimate using MODIS LTA values

It seems like this behavior loses important information about the product (that you might be able to get better atmospheric correction later if MOD08 FTP gets less busy).

The meta data should indicate that LTA value was used due to temporary MOD08 un-availability. A framework of handling provisional products could either be integrated into gips or left to the user. Alternatively, temporary unavailablity could be made a fatal error for processing?

@matthewhanson thoughts? @bhbraswell please add $0.02.

icooke@nile:~/src/optis$ python optis2.py
{'cdl_year': '2015',
 'products': ['ndti', 'crc', 'ndvi', 'acca', 'lswi'],
 'site': '/titan/projects/tillage/Indiana/shapefiles/Hendricks_utm16N.shp'}
Retrieving inventory for site Hendricks_utm16N-0
Processing [acca crc lswi ndti ndvi] on 18 dates (23 files)
Extracting LE70210322015071EDC00_MTL.txt
LE70210322015071EDC00_B1: read in 0:00:23.903200
Running atmospheric model (6S)
Retrieving inventory for site tiles
MOD08: fetch tile  for 2015-03-12 00:00:00
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/gips/data/core.py", line 365, in fetch_ftp
    ftp = ftplib.FTP(ftpurl)
  File "/usr/lib/python2.7/ftplib.py", line 120, in __init__
    self.connect(host)
  File "/usr/lib/python2.7/ftplib.py", line 138, in connect
    self.welcome = self.getresp()
  File "/usr/lib/python2.7/ftplib.py", line 222, in getresp
    raise error_temp, resp
error_temp: 421 Too many users are already logged on.  Please try again in 10 minutes.

Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/gips/data/core.py", line 794, in fetch
    cls.Asset.fetch(a, t, d)
  File "build/bdist.linux-x86_64/egg/gips/data/aod.py", line 119, in fetch
    cls.fetch_ftp(asset, tile, date)
  File "build/bdist.linux-x86_64/egg/gips/data/core.py", line 380, in fetch_ftp
    raise Exception("Error downloading: %s" % e)
Exception: Error downloading: 421 Too many users are already logged on.  Please try again in 10 minutes.

Problem fetching asset: Error downloading: 421 Too many users are already logged on.  Please try again in 10 minutes.
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/gips/data/aod.py", line 248, in get_aod
    img = inv[date].tiles[''].open('aod')
  File "build/bdist.linux-x86_64/egg/gips/inventory.py", line 47, in __getitem__
    return self.data[date]
KeyError: datetime.date(2015, 3, 12)

ltad071[]: read (93,48)-(95,50) in 0.016112 seconds
ltad071[]: read (93,48)-(95,50) in 3.176e-06 seconds
AOD: LTA-Daily = 0.114, 0.000676
lta[]: read (93,48)-(95,50) in 0.000148393 seconds
lta[]: read (93,48)-(95,50) in 2.997e-06 seconds
AOD: LTA = 0.106537, 0.115884
AOD: Source = Weighted estimate using MODIS LTA values Value = 0.113956725019
  Band        T      Lu      Ld
  BLUE:    0.993   33.33  361.22
 GREEN:    0.960   16.26  339.66
   RED:    0.966    8.10  300.32
   NIR:    0.959    2.85  216.66
 SWIR1:    0.978    0.09   47.22
 SWIR2:    0.954    0.01   16.02
Ran atmospheric model in 0:00:10.626315
GIPPY: ACCA - LE70210322015071EDC00_B1
LE70210322015071EDC00_B3[RED]: read (0,0)-(8100,3105) in 1.57879 seconds
LE70210322015071EDC00_B2[GREEN]: read (0,0)-(8100,3105) in 1.53035 seconds
^C^C^CLE70210322015071EDC00_B4[NIR]: read (0,0)-(8100,3105) in 1.53258 seconds
^Z