Timeout when doing big mosaic

[jradavenport]

I ran:

$ k2mosaic tpflist 12 22 > tpf_c12m22.txt
$ k2mosaic mosaic tpf_c12m22.txt

This was to make a mosaic of (I thought) all exposures from Module 22 in C12.

After approx 40 min of running (about 60% complete) it crashed and said:

... error I lost
<urlopen error timed out>

Obviously I lost the actual error log when I closed my window.... I am trying to reproduce this error now.

[jradavenport]

Tried again w/ module 23, crashed even sooner (a few min of running). Full boring Traceback below

Traceback (most recent call last):
  File "/Users/davenpj3/anaconda/envs/python2/bin/k2mosaic", line 11, in <module>
    sys.exit(k2mosaic())
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/k2mosaic/ui.py", line 105, in mosaic
    k2mosaic_mosaic(tpf_filenames, cadencenumbers=cadence, step=step)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/k2mosaic/ui.py", line 62, in k2mosaic_mosaic
    mosaic.add_tpf(tpf)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/k2mosaic/core.py", line 72, in add_tpf
    tpf = fits.open(tpf_filename)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/astropy/io/fits/hdu/hdulist.py", line 166, in fitsopen
    lazy_load_hdus, **kwargs)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/astropy/io/fits/hdu/hdulist.py", line 404, in fromfile
    lazy_load_hdus=lazy_load_hdus, **kwargs)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/astropy/io/fits/hdu/hdulist.py", line 1015, in _readfrom
    fileobj = _File(fileobj, mode=mode, memmap=memmap, cache=cache)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/astropy/utils/decorators.py", line 493, in wrapper
    return function(*args, **kwargs)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/astropy/io/fits/file.py", line 123, in __init__
    self.name = download_file(fileobj, cache=cache)
  File "/Users/davenpj3/anaconda/envs/python2/lib/python2.7/site-packages/astropy/utils/data.py", line 1103, in download_file
    raise urllib.error.URLError(e)
urllib2.URLError: <urlopen error timed out>

[barentsen]

Thank you @jradavenport. This may be related to MAST experiencing high load. It looks like we will have to make k2mosaic more robust by increasing the time out duration and/or making sure retry attempts are made.