Add helm charts

[zoriya]

Feature description

Needed steps (to allow every service to scale):

• [x] Move sql migrations code out of the API (#358)
• [ ] See how to handle meilisearch schema
• [x] Split scanner fs-monitor and item scan (use rabbitmq work queue to sync them) (#410)
• [ ] Distribute the scanner cache between instances (probably by using redis)
• [x] Use a named queue for autosync (right now, I think that two instances of autosync would try to update the external service twice) (#435)
• [ ] Add a route to the transcoder to inform the user has left (stop user's transcode when switching instance) (or use rabbimq to tell others instances that a user switched instance)
• [ ] Route users to the same transcoder instance if they are requesting an video already being handled by an instance
• [x] Use traefik as a reverse proxy (#450)

[bo0tzz]

For the immich helm chart, I used the common-library chart from bjw-s as a base, which helped get a chart together much quicker.

[joryirving]

Something to consider with the helm chart, would be to be able to spawn transcoders as needed. Rather than 1 container doing all the transcoding, you could create 1 pod per transcode stream.

[onedr0p]

Rather than 1 container doing all the transcoding, you could create 1 pod per transcode stream.

That may require a Kubernetes operator or for the application to at least be Kubernetes aware and have proper Kubernetes RBAC setup for the app to create transcoding pods. It would be neat but might be outside the scope of this issue.

[bo0tzz]

I'm not particularly familiar with HorizontalPodAutoscalers or KEDA, but maybe if one of those was involved scaling could be done without many changes on Kyoo's side?

[zoriya]

I do want to support transcoder scaling and distributed transcodes. Not sure what is needed for that, I just finished reading the getting started of k8s x)

[onedr0p]

Feel free to reach out if you need any help, I joined your discord 😄

[atmosx]

Rather than 1 container doing all the transcoding, you could create 1 pod per transcode stream.

That may require a Kubernetes operator or for the application to at least be Kubernetes aware and have proper Kubernetes RBAC setup for the app to create transcoding pods. It would be neat but might be outside the scope of this issue.

For HPA an operator is not required. You can setup auto-scaling based on CPU/RAM metrics, which come out of the box with any modern k8s distribution. Auto-scaling on custom metrics (e.g. a combination of transcoding jobs + queue + ) is possible, but requires something like prometheus. Now, people running kubernetes at home (like myself) usually are familiar with this stack (I do this for a living) but I believe it's an overkill. Most home deployments will have a dedicated RPi4/5 or something another low end box running Kyoto (or any media server) with additional storage attached.

An operator handles way more than autoscaling (e.g. automated DB backups/restore, version upgrades, storage expansion, maintenance tasks, etc.)

[MegaShinySnivy]

I'd be willing to help test this chart and by extension Kyoo itself--I have enough users to put it through home/small event use (10ish users)

[zoriya]

Thanks! I started working on it on the feat/helm branch, but I was focused on something else, so It kinda stalled. An app-template based helm charts has been made by the folks in the home-assistant discord server.

I probably won't continue working on the helm chart soon, If someone wants to upstream the helm chart I'm open for it!