search

mediacms-io / mediacms

MediaCMS is a modern, fully featured open source video and media CMS, written in Python/Django and React, featuring a REST API.
https://mediacms.io
GNU Affero General Public License v3.0
2.75k stars 505 forks source link

Scalability question #864

Closed JimmyPun610 closed 1 year ago

JimmyPun610 commented 1 year ago

Hi team,

Thank you for the great work, really appreciate it. I would like to know the scalability about this solution. According to the system layout, can I assume that we could scale it by

  1. Horizontally scale mediacms_web
  2. Horizontally scale postgres database, I noticed that it is singleton instance for the database, any reason of it?
  3. Vertically scale file store

Do we need to handle others server (celery_beat, redis, celery_worker and migrations) image

mgogoulos commented 1 year ago

All files are uploaded by the app (mediacms_web), getting transcoded by celery_worker, and eventually get served by mediacms_web (via nginx)

DB: can be set as an external service (eg RDS) so the scaling takes place there. Filestorage: all services/containers need to have access to the same filestorage, so it has to be persistent. It could be refactored to store files externally but the default behavior requires persistent storage for all celery_worker/mediacms_web

As long as this is happening, you can have more mediacms_web and celery_worker containerrs running (to handle serving of media and actions as transcoding)

celery_beat: no need to scale, can be a single instance redis: is used by the workers (celery_worker) and has to be a single instance, or could be scaled (run as cluster) but this complicates things and I'm not sure is needed (unless there are very high needs for high availability) migrations: they will just run once, when there are new deployments + there are new migrations, and then exit. No need to scale