Caltech-IPAC / hiresprv

Keck HIRES PRV pipeline service access
4 stars 0 forks source link

Directory Listing Truncated #43

Closed bjfultn closed 4 years ago

bjfultn commented 4 years ago

The directory_listing command fails, e.g.:

data = Download('prv.cookies', './', debug=True)

listing = data.directory_listing()

The error is:

JSONDecodeError                           Traceback (most recent call last)
<ipython-input-1-4bee628d39a9> in <module>
      5 data = Download('prv.cookies', './', debug=True)
      6 
----> 7 listing = data.directory_listing()
      8 prtstring = json.dumps(listing, indent=4, sort_keys=True)[:2000]
      9 prtstring += "\n... output truncated ..."

~/anaconda3/envs/hiresprv/lib/python3.7/site-packages/hiresprv/download.py in directory_listing(self)
    179         if self.status == 'ok':
    180 
--> 181             retval = json.loads(self.response.text)
    182 
    183             # hide undocumented files directory containing IDL save files

~/anaconda3/envs/hiresprv/lib/python3.7/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    346             parse_int is None and parse_float is None and
    347             parse_constant is None and object_pairs_hook is None and not kw):
--> 348         return _default_decoder.decode(s)
    349     if cls is None:
    350         cls = JSONDecoder

~/anaconda3/envs/hiresprv/lib/python3.7/json/decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

~/anaconda3/envs/hiresprv/lib/python3.7/json/decoder.py in raw_decode(self, s, idx)
    351         """
    352         try:
--> 353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
    355             raise JSONDecodeError("Expecting value", s, err.value) from None

JSONDecodeError: Unterminated string starting at: line 206817 column 1 (char 5910503)

If I manually go to the CGI URL for the listing command that it's hitting I can see that the output is being truncated and this is causing the problem.

bjfultn commented 4 years ago

This appears to be fixed now.