pabloromeo
commented
3 years ago Hi! Yeah, I run all three parts using Docker Compose (on a Swarm), with a file similar to yours. (However, notice that I'm using LinuxServer Plex images as base, and extending them with the ClusterPlex Dockermods). You would probably want to switch the "experimental" label for a stable release, as I tend to try crazy things out in the experimental tag.
Here's mine:
version: '3.8'
services:
plex:
image: ghcr.io/linuxserver/plex:latest
deploy:
mode: replicated
replicas: 1
update_config:
order: stop-first
placement:
constraints:
- node.labels.type == v4
resources:
reservations:
memory: 350M
limits:
cpus: '3'
environment:
DOCKER_MODS: "ghcr.io/pabloromeo/clusterplex_dockermod:experimental"
VERSION: docker
PUID: 1000
PGID: 1000
TZ: America/Argentina/Buenos_Aires
ORCHESTRATOR_URL: http://plex-orchestrator:3500
PMS_IP: 192.168.2.1 #plex
TRANSCODE_OPERATING_MODE: both #(local|remote|both)
TRANSCODER_VERBOSE: "1"
healthcheck:
test: curl -fsS http://localhost:32400/identity > /dev/null || exit 1
interval: 30s
timeout: 15s
retries: 5
start_period: 120s
networks:
- web
volumes:
- /mnt/volume1/cluster-data/plex:/config
- /mnt/volume1/cluster-data/plex-tmp:/tmp
- /mnt/nfs/cluster-shared/cluster-backups/plex:/backups
- /media/MediaLibraries:/data
- /etc/localtime:/etc/localtime:ro
ports:
- 32469:32469
- 32400:32400
- 32401:32401
- 3005:3005
- 8324:8324
- 1900:1900/udp
- 32410:32410/udp
- 32412:32412/udp
- 32413:32413/udp
- 32414:32414/udp
logging:
driver: loki:latest
options:
loki-url: "http://localhost:3100/loki/api/v1/push"
plex-orchestrator:
image: ghcr.io/pabloromeo/clusterplex_orchestrator:experimental
deploy:
mode: replicated
replicas: 1
update_config:
order: start-first
resources:
reservations:
memory: 50M
healthcheck:
test: curl -fsS http://localhost:3500/health > /dev/null || exit 1
interval: 15s
timeout: 15s
retries: 5
start_period: 30s
environment:
TZ: America/Argentina/Buenos_Aires
STREAM_SPLITTING: "OFF" # ON | OFF (default)
LISTENING_PORT: 3500
WORKER_SELECTION_STRATEGY: "LOAD_RANK" # RR | LOAD_CPU | LOAD_TASKS | LOAD_RANK
networks:
- web
volumes:
- /etc/localtime:/etc/localtime:ro
ports:
- 3500:3500
logging:
driver: loki:latest
options:
loki-url: "http://localhost:3100/loki/api/v1/push"
plex-worker:
image: ghcr.io/linuxserver/plex:latest
hostname: "plex-worker-{{.Node.Hostname}}"
deploy:
mode: replicated
replicas: 2
placement:
max_replicas_per_node: 1
constraints:
- node.labels.transcoding == true
update_config:
order: stop-first
resources:
reservations:
cpus: '0.25'
memory: 100M
limits:
cpus: '3'
memory: 2048M
environment:
DOCKER_MODS: "ghcr.io/pabloromeo/clusterplex_worker_dockermod:experimental"
VERSION: docker
PUID: 1000
PGID: 1000
TZ: America/Argentina/Buenos_Aires
LISTENING_PORT: 3501
STAT_CPU_INTERVAL: 2000
ORCHESTRATOR_URL: http://plex-orchestrator:3500
healthcheck:
test: curl -fsS http://localhost:3501/health > /dev/null || exit 1
interval: 30s
timeout: 15s
retries: 5
start_period: 240s
networks:
- web
volumes:
- /mnt/volume1/cluster-data/plex-codecs:/codecs
- /mnt/volume1/cluster-data/plex-tmp:/tmp
- /media/MediaLibraries:/data
- /etc/localtime:/etc/localtime:ro
logging:
driver: loki:latest
options:
loki-url: "http://localhost:3100/loki/api/v1/push"
networks:
web:
external: trueI believe the key factor here, is that both Plex and the Worker are sharing the container's real /tmp directory.
- /mnt/volume1/cluster-data/plex-tmp:/tmpThat is a network share I share through GlusterFS. And in Plex, the value for the Transcode directory is:
Also, as you can see there, the Workers don't even need the full Plex config files, it just needs access to the Media in the same path, the temp directory used for transcoding, and the codecs (these don't necessarily need to be shared, i just share them so that they are only downloaded once and reused by all workers).
You may want to enable debug logging in the Plex Server, just to see if you are actually seeing the calls back being made by the Transcoder to your original PMS to report Progress (for example, posting to "-progressurl","http://10.100.10.1:32400/video/:/transcode/session/xf4ofurs5sj3hwfvbn81edps/d373fcf8-f964-4000-8949-b3d9a07586ba/progress").
I believe if you were to share the entire /tmp between them, your problem should go away, because Plex writes stuff into temp aside from just the /tmp/transcode Session content. If you look at your worker's /tmp directory you're probably going to see stuff there. And that might be what is causing the failures on your deployment.
In my case I've loosened permissions on that temp location quite a bit, i've chmod'ed it to +777, which at this time I can't recall if that is completely necessary.
lkathke
Hello guys, I am trying now since 6 hours to get the cluster as simple as possible to work, but i doesnt.
Currently I am using this docker-compose.yml (with docker hub image *-amd64-1.2.9): docker-compose.yml.txt
After starting a movie, plex sends the command to the orchestrator:
So the cwd (current working directory) is: /config/Library/Application Support/Plex Media Server/Cache/Transcode/Sessions/plex-transcode-xf4ofurs5sj3hwfvbn81edps-d373fcf8-f964-4000-8949-b3d9a07586ba which is also accessible through worker1, because I'm sharing the config directory with plex and worker1 /docker/plex/live/config:/config
Worker1 is able to start transcoding and outputs its files to /config/Library/Application Support/Plex Media Server/Cache/Transcode/Sessions/plex-transcode-xf4ofurs5sj3hwfvbn81edps-d373fcf8-f964-4000-8949-b3d9a07586ba
(just an example, not that plex-transcode-xf4of... directory)
But the files dont get published by plex to the webbrowser
Now I read that we should change the transcoder output to /tmp/transcode or /transcode via the plex ui https://github.com/pabloromeo/clusterplex/issues/41#issuecomment-826893413 But this changes the cwd to "/run/s6/services/plex", so the worker can not start to transcode (the directory does not exists, even it would exist. it would be written to that directory, not to /tmp/transcode
Does someone has an idea, how to get it working? Or does someone has a working docker-compose for testing?