clawpack / clawutil

General utility programs
BSD 3-Clause "New" or "Revised" License
10 stars 31 forks source link

Fix logic for forcing over writing of downloaded files #123

Closed mandli closed 6 years ago

mandli commented 7 years ago

Turns out that files were not overwritten properly if they were not archives.

rjleveque commented 7 years ago

For reasons I don't understand, this throws the error below while the version on master works fine when executing $CLAW/geoclaw/examples/tsunami/chile2010/maketopo.py.

[chile2010] D-10-18-240-247 $ rm /Users/rjl/git/clawpack/geoclaw/scratch/etopo10min120W60W60S0S.asc
remove /Users/rjl/git/clawpack/geoclaw/scratch/etopo10min120W60W60S0S.asc? y

[chile2010] D-10-18-240-247 $ python maketopo.py
Downloading http://www.geoclaw.org/topo/etopo/etopo10min120W60W60S0S.asc to /Users/rjl/git/clawpack/geoclaw/scratch/etopo10min120W60W60S0S.asc...
Traceback (most recent call last):
  File "maketopo.py", line 112, in <module>
    get_topo(False)
  File "maketopo.py", line 33, in get_topo
    file_name=topo_fname, verbose=True)
  File "/Users/rjl/git/clawpack/clawpack/clawutil/data.py", line 108, in get_remote_file
    remote_file = urlopen(url)
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 429, in open
    response = self._open(req, data)
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 447, in _open
    '_open', req)
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1228, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1201, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1136, in getresponse
    response.begin()
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
socket.error: [Errno 54] Connection reset by peer
mandli commented 7 years ago

I have not been able to reproduce this even on a slow connection. The only thing I can think of is that the somehow the master branch version is not downloading the file and when it tries it's taking too long.

mandli commented 7 years ago

I just added the ability to unzip zip files as well.

rjleveque commented 6 years ago

This seems to work fine now.