Giant change to add in a worker queue for video uploads. Highlights include:
Videos no longer upload synchronously! Instead they get added to a worker queue processed by Graphile Worker
Two new containers are required when running Match Uploader, one for the worker and the other for a Postgres database
The worker communicates events to the server through websockets (using Socket.IO). This allows the server to inform the web client of worker job status changes in real-time
This solution leaves out a number of worker features that may be necessary in the future, in the interest of getting an MVP completed. Some notable omissions include:
Minimal support to view past completed/failed jobs unless you have Match Uploader open when the job fails
Incomplete implementation of utility API routes to handle stuck worker jobs
No warnings in the UI to catch when the worker queue has backed up (e.g., because the worker is stopped)
Breaking change footer for squash commit:
BREAKING CHANGE: Docker Compose configuration for production use cases requires two new containers for the worker and database containers. See docker-compose.yml for an updated version.
TODO:
[x] Finish validating Docker Compose updates
[x] Confirm Docker Compose setup works when starting from scratch (i.e., creating production.env file and running docker compose up are the only 2 steps to get going)
[x] Update readme with new info about Docker Compose setup
[x] Prepare release notes for updating existing setups to migrate to this version
Giant change to add in a worker queue for video uploads. Highlights include:
This solution leaves out a number of worker features that may be necessary in the future, in the interest of getting an MVP completed. Some notable omissions include:
Breaking change footer for squash commit:
TODO:
production.env
file and runningdocker compose up
are the only 2 steps to get going)Closes #66