sometimes fails with JSONDecodeError

[sylvainfaivre]

🐛 Bug Report

Sometimes (I don't know what triggers this), pwpush-cli aborts with this error :

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

🔬 How To Reproduce

Running with a self hosted AWS instance.

I have this code running inside a for loop :

    tokens[$user]=$(pwgen -scn 32 1)
    echo "--------------------------------------------------------------------------------------------------------"
    echo "Running command : ${PWPUSH} push ${tokens[$user]}"
    echo "--------------------------------------------------------------------------------------------------------"
    urls[$user]=$(${PWPUSH} push ${tokens[$user]})
    ret=$?
    echo "Return code : $ret"
    if (( $ret != 0 )); then
      echo "Exiting due to pwpush error !"
      exit 1
    fi
    echo "--------------------------------------------------------------------------------------------------------"
    echo "Target: $target, User: $user, team: ${teams[$user]}, email: ${emails[$user]}, token: ${tokens[$user]}, url: ${urls[$user]}"
    echo "--------------------------------------------------------------------------------------------------------"

This code sometimes fails with the following stack trace :

--------------------------------------------------------------------------------------------------------
Running command : /REDACTED/venv/bin/pwpush push 1Il7zgWTYWZ9GifG1JH65Q12itRgN4i2
--------------------------------------------------------------------------------------------------------
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /REDACTED/venv/lib/python3.11/site-packages/requests/models.p │
│ y:971 in json                                                                                    │
│                                                                                                  │
│    968 │   │   │   │   │   raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)                    │
│    969 │   │                                                                                     │
│    970 │   │   try:                                                                              │
│ ❱  971 │   │   │   return complexjson.loads(self.text, **kwargs)                                 │
│    972 │   │   except JSONDecodeError as e:                                                      │
│    973 │   │   │   # Catch JSON-related errors and raise as requests.JSONDecodeError             │
│    974 │   │   │   # This aliases json.JSONDecodeError and simplejson.JSONDecodeError            │
│                                                                                                  │
│ /usr/lib/python3.11/json/__init__.py:346 in loads                                                │
│                                                                                                  │
│   343 │   if (cls is None and object_hook is None and                                            │
│   344 │   │   │   parse_int is None and parse_float is None and                                  │
│   345 │   │   │   parse_constant is None and object_pairs_hook is None and not kw):              │
│ ❱ 346 │   │   return _default_decoder.decode(s)                                                  │
│   347 │   if cls is None:                                                                        │
│   348 │   │   cls = JSONDecoder                                                                  │
│   349 │   if object_hook is not None:                                                            │
│                                                                                                  │
│ /usr/lib/python3.11/json/decoder.py:337 in decode                                                │
│                                                                                                  │
│   334 │   │   containing a JSON document).                                                       │
│   335 │   │                                                                                      │
│   336 │   │   """                                                                                │
│ ❱ 337 │   │   obj, end = self.raw_decode(s, idx=_w(s, 0).end())                                  │
│   338 │   │   end = _w(s, end).end()                                                             │
│   339 │   │   if end != len(s):                                                                  │
│   340 │   │   │   raise JSONDecodeError("Extra data", s, end)                                    │
│                                                                                                  │
│ /usr/lib/python3.11/json/decoder.py:355 in raw_decode                                            │
│                                                                                                  │
│   352 │   │   try:                                                                               │
│   353 │   │   │   obj, end = self.scan_once(s, idx)                                              │
│   354 │   │   except StopIteration as err:                                                       │
│ ❱ 355 │   │   │   raise JSONDecodeError("Expecting value", s, err.value) from None               │
│   356 │   │   return obj, end                                                                    │
│   357                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /REDACTED/venv/lib/python3.11/site-packages/pwpush/__main__.p │
│ y:177 in push                                                                                    │
│                                                                                                  │
│   174 │   │   path = "/p/%s/preview.json" % body["url_token"]                                    │
│   175 │   │   response = make_request("GET", path)                                               │
│   176 │   │                                                                                      │
│ ❱ 177 │   │   body = response.json()                                                             │
│   178 │   │   if json_output():                                                                  │
│   179 │   │   │   print(body)                                                                    │
│   180 │   │   else:                                                                              │
│                                                                                                  │
│ /REDACTED/venv/lib/python3.11/site-packages/requests/models.p │
│ y:975 in json                                                                                    │
│                                                                                                  │
│    972 │   │   except JSONDecodeError as e:                                                      │
│    973 │   │   │   # Catch JSON-related errors and raise as requests.JSONDecodeError             │
│    974 │   │   │   # This aliases json.JSONDecodeError and simplejson.JSONDecodeError            │
│ ❱  975 │   │   │   raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)                            │
│    976 │                                                                                         │
│    977 │   @property                                                                             │
│    978 │   def links(self):                                                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Return code : 1
Exiting due to pwpush error !

This seems to be related to some kind of rate limit, as it doesn't happen if I add a sleep invocation in the loop.

However, this behaviour differs from the one with the 403 error I posted in another issue.

Environment

• OS: Ubuntu 23.04
• Python 3.11.4

[sylvainfaivre]

I tried adding -d -v options but got no more information.

[pglombardo]

Hi @sylvainfaivre - thanks for reporting. I believe I need to check HTTP response status code before attempting to decode the JSON. I'll take a look at this soon.