FiniteSingularity
commented
2 years ago Here is the problem we're seeing, and I am open to suggestions in terms of how to reduce resource usage for Railway.
- Railway's "free" tier is essentially $5 in credits given each month. The current cost to run an instance of TAU is around $5.50-$6.00, just outside of the free tier.
- Unfortunately, there is no way that I can see to have data persistence if we use the sqlite option for TAU (which would keep us from having to run a Postgres container. The current CPU/Memory usage of the postgres container is about 50% of our overall Railway resource usage).
- We also require a redis container as a message broker for Django channels. The good news on this front- the redis container is a very small portion of the overall resource usage- basically a rounding error.
- Daphne is known to have issues with memory leaks, and you have very little control over its memory/cpu usage (which is known to be quite high, especially for small projects like this).
- Uvicorn is an ASGI server process, much like Daphne, but appears in all ways to be more slim/customizable.
As you might imagine, the average CPU usage of TAU is quite low- it sits almost idle most of the time, and processing incoming events/IRC, and passing them through to a few websocket subscribers is not very resource intensive. The resource usage that is putting us over the $5 credit is memory.
Looking at my last month's usage, the average memory was a few mb for Redis (again, rounding error), ~240mb for Postgres, and 280mb for the server/worker container (running Daphne). It looks like keeping memory usage to under 500mb total would keep us in the free tier.
I've seem some estimates that Uvicorn used for smaller projects (and only using 1 or 2 work threads) can save up to half the memory usage over Daphne. I'd estimate that 50-60mb of our usage is the TAU worker process, and the remaining 200+ is Daphne. So if we can cut that in half, we'd be well under the 500mb limit. I can try to test this out later next week, but if anyone wants to give it a try themselves, you'll need to use pip in your container to install Uvicorn (or add it to the requirements.txt file and rebuild the container), then change the [program:server] command in the supervisord.conf file to run uvicorn (or even gunicorn with a uvicorn worker). I am seeing conflicting issues on if the /tau/asgi.py file will need to be modifed to run under uvicorn.
It would be ideal to stay under the Railway $5 plan, if possible, which is a resource usage-based IaaS provider that TAU supports deployments for. @FiniteSingularity made a suggestion to try Uvicorn instead of Daphne to use less resources.
Additional context