Cannot fetch GISTEMPv3_sources.tar.gz

[GoogleCodeExporter]

What steps will reproduce the problem?
1. python tool/fetch.py

What is the expected output? What do you see instead?

The fetch aborts shortly after fetching a 14 byte file (which it leaves in the 
input directory).

Fetching http://data.giss.nasa.gov/gistemp/sources_v3/GISTEMPv3_sources.tar.gz 
to input/GISTEMPv3_sources.tar.gz
  14/14 [100%]
Traceback (most recent call last):
  File "tool/fetch.py", line 484, in <module>
    sys.exit(main())
  File "tool/fetch.py", line 476, in main
    fetcher.fetch()
  File "tool/fetch.py", line 127, in fetch
    self.fetch_one(url, local, members=members)
  File "tool/fetch.py", line 301, in fetch_one
    self.fetch_url(url, local, members)
  File "tool/fetch.py", line 328, in fetch_url
    self.extract(name, members)
  File "tool/fetch.py", line 351, in extract
    self.extract_tar(name, members)
  File "tool/fetch.py", line 363, in extract_tar
    tar = tarfile.open('', mode='r', fileobj=open(name,'r'))
  File "/usr/lib/python2.7/tarfile.py", line 1665, in open
    raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully

Please use labels and text to provide additional information.

The core problem is that GISS changed to an HTTP server that rejects the 
request unless a User-Agent header is sent (I checked with curl, which 
downloads the file fine, but rejects with 403 if you use -A '' to set the 
User-Agent header to nothing).

Original issue reported on code.google.com by drj...@googlemail.com on 7 Jul 2014 at 7:48

[GoogleCodeExporter]

A workaround is to

cd input
curl -O http://data.giss.nasa.gov/gistemp/sources_v3/GISTEMPv3_sources.tar.gz

which downloads the file "by hand" (the remainder of the fetch process is then 
happy to work with that).

Original comment by drj...@googlemail.com on 7 Jul 2014 at 7:50