Use API with Python: Connection aborted, RemoteDisconnected('Remote end closed connection without response')

[fleetingbytes]

Question

I am trying to use the API with Python's requests library. Following the API documentation how to retrieve an entry I wrote this minimal script:

from requests import request

url = "https://regex101.com/api/regex/AaKDTO/1"
response = request("GET", url)

I expect to get a 404 response with the following content: {"error":"Regex does not exist","errorId":"b20d027a-fb34-4e34-bb4e-c2e60f568b98"}.

However, when I run it, I get this:

Traceback (most recent call last):
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 537, in _make_request
    response = conn.getresponse()
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connection.py", line 466, in getresponse
    httplib_response = super().getresponse()
  File "C:\Program Files\Python310\lib\http\client.py", line 1374, in getresponse
    response.begin()
  File "C:\Program Files\Python310\lib\http\client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "C:\Program Files\Python310\lib\http\client.py", line 287, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\adapters.py", line 486, in send
    resp = conn.urlopen(
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\util\retry.py", line 470, in increment
    raise reraise(type(error), error, _stacktrace)
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\util\util.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connectionpool.py", line 537, in _make_request
    response = conn.getresponse()
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\urllib3\connection.py", line 466, in getresponse
    httplib_response = super().getresponse()
  File "C:\Program Files\Python310\lib\http\client.py", line 1374, in getresponse
    response.begin()
  File "C:\Program Files\Python310\lib\http\client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "C:\Program Files\Python310\lib\http\client.py", line 287, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\fleetingbytes\src\minimal\minimal.py", line 5, in <module>
    res = request("GET", url)
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\fleetingbytes\env\req\lib\site-packages\requests\adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Is anything wrong with the API or the API documentation?

[working-name]

Hi @fleetingbytes

It seems you're bumping into whatever anti-bot/DDOS protection is running in front of /api. I'm not privy to the details, but from testing it seems that if you use a User-Agent other than python's default it should be good to go. Tested it with a browser, insomnia, and curl from cli. As soon as I tell curl to send the header I get the same response as you.

$ curl "https://regex101.com/api/regex/AaKDTO/1"
{"error":"Regex does not exist","errorId":"30eeb9bb-ecb5-4a54-a840-c6711b8bb124"}

$ curl "https://regex101.com/api/regex/AaKDTO/1" -H "User-Agent: python-requests2.31.0"
curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)

[firasdib]

@working-name is right