Internal Server when sending GET request from Android

[ghost]

I currently ran into a strange issue where the I can send GET requests from a browser or curl and receive a totally fine 200 OK,

<ip> - - [26/Jul/2022 19:37:34] "GET /places/18526165 HTTP/1.1" 200 -

but when I send the very same request from my Android app, the server responds with

[2022-07-26 19:36:25,442] ERROR in app: Exception on /places/18526165 [GET]
Traceback (most recent call last):
  File "<path>/venv/lib/python3.10/site-packages/flask/app.py", line 1523, in full_dispatch_request
    rv = self.dispatch_request()
  File "<path>/venv/lib/python3.10/site-packages/flask/app.py", line 1509, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "<path>/venv/lib/python3.10/site-packages/flask_restx/api.py", line 407, in wrapper
    return self.make_response(data, code, headers=headers)
  File "<path>/venv/lib/python3.10/site-packages/flask_restx/api.py", line 438, in make_response
    raise InternalServerError()
werkzeug.exceptions.InternalServerError: 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
<ip> - - [26/Jul/2022 19:36:25] "GET /places/18526165 HTTP/1.1" 500 -

Before adding flask-restx, the server responded correctly with 200 OK. I do not understand, what causes the 500 Internal Server Error.

The Java code used to issue the GET request:

    public String retrieveFromURL(URL url){
        StringBuilder stringBuilder = new StringBuilder();
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream(), StandardCharsets.UTF_8))) {
            for (String line; (line = reader.readLine()) != null; ) {
                stringBuilder.append(line);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return stringBuilder.toString();
    }

I really can't think of why. So I decided to capture HTTP GET on the host running the server:

From Firefox / Swagger UI

root@wow:~# tcpdump -i eth0 -s 0 -A 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
19:51:01.899886 IP <ip> > <ip>: Flags [P.], seq 3108888141:3108888457, ack 3825282718, win 502, options [nop,nop,TS val 1019256520 ecr 2267856041], length 316
E..p`\@.6.~2........{....M.M.........=.....
<....,..GET /places/18526165 HTTP/1.1
Host: <ip:port>
User-Agent: Mozilla/5.0 (redacted)
Accept: application/json
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: <ip:port>
DNT: 1
Connection: keep-alive

Android (500 Internal Server Error):

19:51:40.463262 IP <ip> > <ip>: Flags [P.], seq 2411747994:2411748182, ack 3002101130, win 685, options [nop,nop,TS val 17801935 ecr 2267894550], length 188                                                                   
E....a@.6...........J.....Z...m......t.....                                                              
.....-S.GET /places/18526165 HTTP/1.1                                                                    
User-Agent: Dalvik/2.1.0 (Linux; U; Android 9; SM-J330F Build/PPR1.180610.011)                           
Host: <ip:port>                                                                 
Connection: Keep-Alive                                                                                   
Accept-Encoding: gzip                                                                                    

Curl (200 OK)

19:52:10.512679 IP <ip> > <ip>: Flags [P.], seq 1265862891:1265863012, ack 352242510, win 502, options [nop,nop,TS val 1019325130 ecr 2267924651], length 121                                                                  
E.....@.6..m........J...Ks.....N...........                                                              
<....-..GET /places/18526165 HTTP/1.1                                                           
Host: <ip:port>                                                                    
User-Agent: curl/7.84.0                                                                                  
accept: application/json  

Environment

Python 3.10.5 Flask 2.1.2 Werkzeug 2.1.2 Flask_restx 0.5.1

[ghost]

Even more baffling is, that when I sent the dysfunctional Android request manually using CURL, the server responds just fine to it with 200.

$ curl -X 'GET'   '<ip:port>/places/18526165'   -H 'Accept-Encoding: gzip' -H 'Connection: Keep-Alive' -H 'User-Agent: Dalvik/2.1.0 (Linux; U; Android 9; SM-J330F Build/PPR1.180610.011'

Resulting dump:

Host: <ip:port>
Accept: */*
Accept-Encoding: gzip
Connection: Keep-Alive
User-Agent: Dalvik/2.1.0 (Linux; U; Android 9; SM-J330F Build/PPR1.180610.011

[peter-doggart]

It looks like the android request is maybe setting an accept header that the flask-restx app doesn't support.

The bit of code in question in api.py is:

mediatype = request.accept_mimetypes.best_match(
            self.representations, default=default_mediatype,
        )

Can you manually set the accept: http header in your android request to application/json?

[ghost]

Hello Peter, thanks for the answer. I first thought that I created the very same GET-request using curl, but now I tcpdumped it and saw thaw that curl sends default headers I was not aware of.

Indead, you found the problem, I can reproduce it using curl now:

$ curl -X 'GET'   '<url>'   -H 'Accept-Encoding: gzip' -H 'Connection: Keep-Alive' -H 'User-Agent: Dalvik/2.1.0 (Linux; U; Android 9; SM-J330F Build/PPR1.180610.011' -H 'Accept:'
<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

Thanks, so all I got to do is add an Accept header. :) You just saved my day.

I would recommend adjusting flask_restx to either throwing a more explicit error message or allowing to remove the header when sending a request as this is no behavior someone with rudimentary understanding of HTTP would expect ;)