500 Internal Server Errors

[sirfz]

I'm using mod_wsgi with mod_wsgi-httpd and I'm seeing status 500 in the access logs but after debugging it doesn't seem to be an issue from the python wsgi app. I'm trying to understand what the error is but I can't find any log files for the Apache server (other than the ones logging my wsgi app which contain no errors). Is there any log file generated by mod_wsgi-httpd that I can check for errors?

[GrahamDumpleton]

I presume you must be using mod_wsgi-express? One can't use mod_wsgi-httpd package without using it really.

In that case, when you start mod_wsgi-express it will tell you the location of the error log file generated by Apache. This will include any Python exceptions generated from your WSGI application which propagate all the way up to the WSGI server level.

If having trouble finding that, you can use the --log-to-terminal option to mod_wsgi-express and both error log, and access log if enabled, will be output to the console where you ran mod_wsgi-express.

If there is nothing in that error log output about any exception, then it may mean that your WSGI framework that you are using is catching the exception and translating that to a generic HTTP 500 response. In this case it is generally necessary to configure the WSGI framework to log the details of the error, or otherwise report it somehow.

For example, if you were using Django, see:

• http://blog.dscpl.com.au/2015/04/integrating-modwsgi-express-as-django.html

for details of configuring LOGGING in Django settings module.

If not Django, you would need to say what WSGI framework you are using.

Also indicate whether you see the 500 error response in the browser and if so whether it is a WSGI framework generated page or more likely the standard Apache one.

Also show the actual access log snippet so can see what URL it is and whether it is valid WSGI application URL or one a magic URL such as favicon.ico that browsers will request. If a WSGI application doesn't support that, usually would get a 404, but depends on how your URL mapping in WSGI application is setup.

So, provide more details on how you are running mod_wsgi and whether using mod_wsgi-express and with what options, plus what WSGI application/framework.

If have a better idea of what you are using, then can if necessary explain some debug modes that can be used to capture request and response to work out where the 500 response is coming from.

[sirfz]

Sorry for the late reply. Just got back to this issue. Yes, I am using mod_wsgi-express but my problem is that I don't see the Apache-specific error (which I made sure as you suggested was not an error returned by my wsgi app). I'm using bottle wsgi framework.

It seems my issue is connection timeouts where my application is unable to handle all the requests its receiving. I got to this conclusion after reducing the queue-timeout where I started seeing lots of 504s. My conclusion was that the client was closing the connection from their side sooner than the wsgi app was able to handle the request and that probably caused internal server errors on Apache. But unfortunately I was not seeing any errors in the logs, why is that?

[GrahamDumpleton]

Quite possible that relates to backlogging of requests and subsequent disconnection by a client. The issue is discussed in:

• https://github.com/GrahamDumpleton/mod_wsgi/issues/75

Unfortunately, specifically identifying the cause and logging a more specific error message isn't really possible.

If you are reaching capacity and getting backlogging, then you would need to look at how many processes/threads are being provisioned and whether that needs to be increased to accommodate long running requests. The default is only a single process with 5 threads. For very quick requests this can handle quite a lot of requests, but long running requests would be a problem.

If you aren't generally expecting long running requests, then issue could still be a slow down in communicating with backend services such as databases, external web services etc. A blockage there will mean the request thread is tied up longer than expected and so reduces capacity which can then result in backlog if you start getting an influx of requests.

Do you have any idea of what typical response time should be and what your throughput is? Are you using the default single process with 5 threads or have you overridden that using the --processes and --threads options.

[sirfz]

I currently have --processes 1 --threads 1 (it was just a test instance but it's being heavily used - 50 requests / second), I don't want to increase the number of threads right now because my processing class is not thread-safe yet and it wouldn't make much difference anyway since my app spends all its time processing (no IO involved). I increased the number of processes up to 8 after making sure they'll fit in memory but that wasn't enough. That's not mod_wsgi's fault of course. I need to drastically reduce the response time of my app which is the main bottleneck. My main concern at first was trying to understand where the errors are coming from but I think it's pretty clear now. Thank you for your help.

[GrahamDumpleton]

Okay. I will close this now then. If you need to discuss further then I would suggest the mod_wsgi mailing list as perhaps a better forum.