Blank pages

[alexei]

I started evaluating granian to run a Django project. The site currently runs on gunicorn, so that's my baseline. I ran benchmarks using wrk against a couple of pages, and the results are promising.

However manual testing revealed a worrying situation: some pages are blank.

To give you an example, there's a login page where, after submitting their credentials, the user is authenticated and then redirected to their dashboard. With granian, the user is not authenticated, and the redirect doesn't happen.

It happens to simpler pages as well. For example, accessing the admin root page might result in a blank page.

Note these pages are not particularly special.

To make things weirder, upon restarting the server, it is possible that a page that was blank earlier now works. Or the other way around.

I am unable to reproduce it locally, and server logs show nothing unusual. For example, when the homepage works as expected, but the admin results in an empty page, the logs are as follows:

[time] heroku[router]: at=info method=GET path="/" host=my-project.com request_id=... fwd="..." dyno=web.1 connect=0ms service=59ms status=200 bytes=30274 protocol=https
[time] heroku[router]: at=info method=GET path="/admin" host=my-project.com request_id=... fwd="..." dyno=web.1 connect=0ms service=58ms status=301 bytes=194 protocol=https
[time] heroku[router]: at=info method=GET path="/admin/" host=my-project.com request_id=... fwd="..." dyno=web.1 connect=0ms service=9ms status=200 bytes=106 protocol=https

For what it's worth, I'm running both apps on basic Heroku dynos with the default settings. Python is 3.11, Django is 4.2, granian is 1.0.2, while gunicorn is 21.2.0.

gunicorn runs as:

gunicorn project.wsgi --log-level info

granian too:

granian --interface wsgi --port $PORT --host 0.0.0.0 project.wsgi:application

Currently the env vars look like this:

GRANIAN_LOG_LEVEL=warning
GRANIAN_RESPAWN_FAILED_WORKERS=1
GRANIAN_WORKERS=1

I considered the possibility that the dynos are underpowered (they are!) and switched to premium dynos, but the result was the same. The site pages feel snappy alright, but only when they work. I also played with 2-3 workers, but that didn't help either. GRANIAN_RESPAWN_FAILED_WORKERS doesn't seem to make a difference.

What am I doing wrong? What are some things that I could try?

[gi0baro]

So, I'm not sure I got this correctly.

When you say you get blank pages you mean in the browser? 'cause the strange thing to me is that the logs seems to report a correct response code and also a non-empty content, at least in terms of bytes. Also the fact this is heroku-related only seems quite weird.

Some things that might help debugging the issue would be making requests through something different from browser (maybe a curl in verbose mode), increasing the log-level of granian to debug and disabling the respawn.

Given all the above, I can only suspect there might be something wrong with heroku and memory management, maybe their dynos get restarted while granian is sending back the response.. I can't think of anything else at the moment that can explain this.

[alexei]

I apologise for the delay. Yes, that's exactly what I'm saying and I'm as surprised as you are: when served by granian, the page in the browser is blank. When served by gunicorn, which is the server we currently use in production, it works as expected.

I've opened a ticket with Heroku as well, but I'm having a little trouble convincing them to investigate this. Their support noticed an unrelated error generated in one of the Celery tasks and they've developed a fixation with that 🤦‍♂️ I'll need to address that first before I give it another shot.

I'll be back after I'm done.

[gi0baro]

Closing this due to inactivity