WebSocket Error with Traefik 2

klangborste commented 2 years ago

Hello together,

I followed the documentation https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-docker/

Then used ./gen-passwords.sh and tried to figured out with different docker-compose templates what I must configure to get Jitsi on docker-compose up and running.

This is the error I get at the moment on the browser:

The newest stable docker-jitsi-meet-stable-7210 is used.

The docker-compose.yml snippets from what I changed: this network was added as application network

jitsi_net:
    name: jitsi_net
    driver: bridge
    ipam:
      config:
        - subnet: 192.168.95.0/24

last entryPoint in this snipped was for video over port 10000 added

# Traefik 2 - Reverse Proxy
  traefik:
    container_name: traefik
    image: traefik:latest
    restart: always
    command: # CLI arguments
      - --global.checkNewVersion=true
      - --global.sendAnonymousUsage=true
      - --entryPoints.http.address=:80
      - --entryPoints.https.address=:443
      - --entryPoints.video.address=:10000

Click here to see the added application config itself to the docker-compose.yml

``` # Jitsi Video Conferencing # Frontend web: container_name: jitsi-web image: jitsi/web restart: unless-stopped volumes: - ${DOCKERDIR}/jitsi/web:/config - ${DOCKERDIR}/jitsi/web/crontabs:/var/spool/cron/crontabs - ${DOCKERDIR}/jitsi/transcripts:/usr/share/jitsi-meet/transcripts environment: - AMPLITUDE_ID - ANALYTICS_SCRIPT_URLS - ANALYTICS_WHITELISTED_EVENTS - CALLSTATS_CUSTOM_SCRIPT_URL - CALLSTATS_ID - CALLSTATS_SECRET - CHROME_EXTENSION_BANNER_JSON - CONFCODE_URL - CONFIG_EXTERNAL_CONNECT - DEFAULT_LANGUAGE - DEPLOYMENTINFO_ENVIRONMENT - DEPLOYMENTINFO_ENVIRONMENT_TYPE - DEPLOYMENTINFO_REGION - DEPLOYMENTINFO_SHARD - DEPLOYMENTINFO_USERREGION - DESKTOP_SHARING_FRAMERATE_MIN - DESKTOP_SHARING_FRAMERATE_MAX - DIALIN_NUMBERS_URL - DIALOUT_AUTH_URL - DIALOUT_CODES_URL - DISABLE_AUDIO_LEVELS - DISABLE_DEEP_LINKING - DISABLE_GRANT_MODERATOR - DISABLE_HTTPS - DISABLE_KICKOUT - DISABLE_POLLS - DISABLE_REACTIONS - DROPBOX_APPKEY - DROPBOX_REDIRECT_URI - DYNAMIC_BRANDING_URL - ENABLE_AUDIO_PROCESSING - ENABLE_AUTH - ENABLE_BREAKOUT_ROOMS - ENABLE_CALENDAR - ENABLE_COLIBRI_WEBSOCKET - ENABLE_FILE_RECORDING_SERVICE - ENABLE_FILE_RECORDING_SERVICE_SHARING - ENABLE_FLOC - ENABLE_GUESTS - ENABLE_HSTS - ENABLE_HTTP_REDIRECT - ENABLE_IPV6 - ENABLE_LETSENCRYPT - ENABLE_LIPSYNC - ENABLE_NO_AUDIO_DETECTION - ENABLE_NOISY_MIC_DETECTION - ENABLE_PREJOIN_PAGE - ENABLE_P2P - ENABLE_WELCOME_PAGE - ENABLE_CLOSE_PAGE - ENABLE_RECORDING - ENABLE_REMB - ENABLE_REQUIRE_DISPLAY_NAME - ENABLE_SIMULCAST - ENABLE_STATS_ID - ENABLE_STEREO - ENABLE_SUBDOMAINS - ENABLE_TALK_WHILE_MUTED - ENABLE_TCC - ENABLE_TRANSCRIPTIONS - ENABLE_XMPP_WEBSOCKET - ETHERPAD_PUBLIC_URL - ETHERPAD_URL_BASE - GOOGLE_ANALYTICS_ID - GOOGLE_API_APP_CLIENT_ID - HIDE_PREMEETING_BUTTONS - INVITE_SERVICE_URL - JICOFO_AUTH_USER - LETSENCRYPT_DOMAIN - LETSENCRYPT_EMAIL - LETSENCRYPT_USE_STAGING - MATOMO_ENDPOINT - MATOMO_SITE_ID - MICROSOFT_API_APP_CLIENT_ID - NGINX_RESOLVER - NGINX_WORKER_PROCESSES - NGINX_WORKER_CONNECTIONS - PEOPLE_SEARCH_URL - PUBLIC_URL - P2P_PREFERRED_CODEC - RESOLUTION - RESOLUTION_MIN - RESOLUTION_WIDTH - RESOLUTION_WIDTH_MIN - START_AUDIO_MUTED - START_AUDIO_ONLY - START_BITRATE - START_SILENT - START_WITH_AUDIO_MUTED - START_VIDEO_MUTED - START_WITH_VIDEO_MUTED - TESTING_CAP_SCREENSHARE_BITRATE - TESTING_OCTO_PROBABILITY - TOKEN_AUTH_URL - TOOLBAR_BUTTONS - TZ - VIDEOQUALITY_BITRATE_H264_LOW - VIDEOQUALITY_BITRATE_H264_STANDARD - VIDEOQUALITY_BITRATE_H264_HIGH - VIDEOQUALITY_BITRATE_VP8_LOW - VIDEOQUALITY_BITRATE_VP8_STANDARD - VIDEOQUALITY_BITRATE_VP8_HIGH - VIDEOQUALITY_BITRATE_VP9_LOW - VIDEOQUALITY_BITRATE_VP9_STANDARD - VIDEOQUALITY_BITRATE_VP9_HIGH - VIDEOQUALITY_ENFORCE_PREFERRED_CODEC - VIDEOQUALITY_PREFERRED_CODEC - XMPP_AUTH_DOMAIN - XMPP_BOSH_URL_BASE - XMPP_DOMAIN - XMPP_GUEST_DOMAIN - XMPP_MUC_DOMAIN - XMPP_RECORDER_DOMAIN networks: - t2_proxy - jitsi_net labels: - "traefik.enable=true" ## HTTP Routers - "traefik.http.routers.jitsi-web.entrypoints=https" - "traefik.http.routers.jitsi-web.rule=Host(`meet.$DOMAINNAME`)" - "traefik.http.routers.jitsi-web.tls=true" - "traefik.http.routers.jitsi-web.service=jitsi-web" ## Middlewares - "traefik.http.routers.jitsi-web.middlewares=chain-no-auth@file" ## HTTP Services - "traefik.http.services.jitsi-web.loadbalancer.server.port=80" - "traefik.docker.network=t2_proxy" # XMPP server prosody: container_name: jitsi-prosody image: jitsi/prosody restart: unless-stopped expose: - '5222' - '5347' - '5280' volumes: - ${DOCKERDIR}/jitsi/prosody/config:/config - ${DOCKERDIR}/jitsi/prosody/prosody-plugins-custom:/prosody-plugins-custom environment: - AUTH_TYPE - DISABLE_POLLS - ENABLE_AUTH - ENABLE_AV_MODERATION - ENABLE_BREAKOUT_ROOMS - ENABLE_GUESTS - ENABLE_LOBBY - ENABLE_XMPP_WEBSOCKET - GLOBAL_CONFIG - GLOBAL_MODULES - JIBRI_RECORDER_USER - JIBRI_RECORDER_PASSWORD - JIBRI_XMPP_USER - JIBRI_XMPP_PASSWORD - JICOFO_AUTH_USER - JICOFO_AUTH_PASSWORD - JICOFO_COMPONENT_SECRET - JIGASI_XMPP_USER - JIGASI_XMPP_PASSWORD - JVB_AUTH_USER - JVB_AUTH_PASSWORD - JWT_APP_ID - JWT_APP_SECRET - JWT_ACCEPTED_ISSUERS - JWT_ACCEPTED_AUDIENCES - JWT_ASAP_KEYSERVER - JWT_ALLOW_EMPTY - JWT_AUTH_TYPE - JWT_TOKEN_AUTH_MODULE - MATRIX_UVS_URL - MATRIX_UVS_ISSUER - MATRIX_UVS_AUTH_TOKEN - MATRIX_UVS_SYNC_POWER_LEVELS - LOG_LEVEL - LDAP_AUTH_METHOD - LDAP_BASE - LDAP_BINDDN - LDAP_BINDPW - LDAP_FILTER - LDAP_VERSION - LDAP_TLS_CIPHERS - LDAP_TLS_CHECK_PEER - LDAP_TLS_CACERT_FILE - LDAP_TLS_CACERT_DIR - LDAP_START_TLS - LDAP_URL - LDAP_USE_TLS - PUBLIC_URL - TURN_CREDENTIALS - TURN_HOST - TURNS_HOST - TURN_PORT - TURNS_PORT - TZ - XMPP_DOMAIN - XMPP_AUTH_DOMAIN - XMPP_GUEST_DOMAIN - XMPP_MUC_DOMAIN - XMPP_INTERNAL_MUC_DOMAIN - XMPP_MODULES - XMPP_MUC_MODULES - XMPP_INTERNAL_MUC_MODULES - XMPP_RECORDER_DOMAIN networks: jitsi_net: aliases: - ${XMPP_SERVER} # Focus component jicofo: container_name: jitsi-jicofo image: jitsi/jicofo restart: unless-stopped volumes: - ${DOCKERDIR}/jitsi/jicofo:/config environment: - AUTH_TYPE - BRIDGE_AVG_PARTICIPANT_STRESS - BRIDGE_STRESS_THRESHOLD - ENABLE_AUTH - ENABLE_AUTO_OWNER - ENABLE_CODEC_VP8 - ENABLE_CODEC_VP9 - ENABLE_CODEC_H264 - ENABLE_OCTO - ENABLE_RECORDING - ENABLE_SCTP - ENABLE_AUTO_LOGIN - JICOFO_AUTH_USER - JICOFO_AUTH_PASSWORD - JICOFO_ENABLE_BRIDGE_HEALTH_CHECKS - JICOFO_CONF_INITIAL_PARTICIPANT_WAIT_TIMEOUT - JICOFO_CONF_SINGLE_PARTICIPANT_TIMEOUT - JICOFO_ENABLE_HEALTH_CHECKS - JICOFO_SHORT_ID - JICOFO_RESERVATION_ENABLED - JICOFO_RESERVATION_REST_BASE_URL - JIBRI_BREWERY_MUC - JIBRI_REQUEST_RETRIES - JIBRI_PENDING_TIMEOUT - JIGASI_BREWERY_MUC - JIGASI_SIP_URI - JVB_BREWERY_MUC - MAX_BRIDGE_PARTICIPANTS - OCTO_BRIDGE_SELECTION_STRATEGY - SENTRY_DSN="${JICOFO_SENTRY_DSN:-0}" - SENTRY_ENVIRONMENT - SENTRY_RELEASE - TZ - XMPP_DOMAIN - XMPP_AUTH_DOMAIN - XMPP_INTERNAL_MUC_DOMAIN - XMPP_MUC_DOMAIN - XMPP_RECORDER_DOMAIN - XMPP_SERVER depends_on: - prosody networks: - jitsi_net # Video bridge jvb: container_name: jitsi-jvb image: jitsi/jvb restart: unless-stopped ports: - '10000:10000/udp' volumes: - ${DOCKERDIR}/jitsi/jvb:/config environment: - DOCKER_HOST_ADDRESS - ENABLE_COLIBRI_WEBSOCKET - ENABLE_OCTO - JVB_AUTH_USER - JVB_AUTH_PASSWORD - JVB_BREWERY_MUC - JVB_PORT - JVB_MUC_NICKNAME - JVB_STUN_SERVERS - JVB_OCTO_BIND_ADDRESS - JVB_OCTO_PUBLIC_ADDRESS - JVB_OCTO_BIND_PORT - JVB_OCTO_REGION - JVB_WS_DOMAIN - JVB_WS_SERVER_ID - PUBLIC_URL - SENTRY_DSN="${JVB_SENTRY_DSN:-0}" - SENTRY_ENVIRONMENT - SENTRY_RELEASE - COLIBRI_REST_ENABLED - SHUTDOWN_REST_ENABLED - TZ - XMPP_AUTH_DOMAIN - XMPP_INTERNAL_MUC_DOMAIN - XMPP_SERVER depends_on: - prosody networks: - jitsi_net labels: - "traefik.enable=true" ## HTTP Routers - "traefik.http.routers.jitsi-jvb.entrypoints=video" #- "traefik.http.routers.jitsi-jvb.tls=true" - "traefik.http.routers.jitsi-jvb.service=jitsi-jvb" ## Middlewares - "traefik.http.routers.jitsi-jvb.middlewares=chain-no-auth@file" ## HTTP Services - "traefik.http.services.jitsi-jvb.loadbalancer.server.port=10000" - "traefik.docker.network=t2_proxy" ```

Thanks in advance for any help :-)

saghul commented 2 years ago

Are you proxying the WS connections correctly? Sorry, I don't know much about Traefik myself. Maybe you can check the community provided examples here: https://github.com/jitsi-contrib/jitsi-traefik

klangborste commented 2 years ago

I used this template https://github.com/jitsi-contrib/jitsi-traefik/blob/main/traefik-v2/docker-compose.yml Did also a lot of trial & error, but I could'nt get any connection to the video bridge. I got also inspiration from https://github.com/jitsi/docker-jitsi-meet/issues/1113#issuecomment-1073432941

Is this maybe a problem that the server tries to connect to localhost, but I am connected to the external world?

saghul commented 2 years ago

Looks like your PUBLIC_URL env var is not properly set.

klangborste commented 2 years ago

You are right, I found it also in the meantime. I checked this before in my analysis but I dont know what I searched there in the .env at that time 😅

The first login works now, but not the transfer from the video and when somebody joined as 3rd person then it crashs again. I guess this needs another issue.

saghul commented 2 years ago

That suggests a problem with the JVB connection. What do you see in the console logs with a 3rd participant joins?

klangborste commented 2 years ago

I edited my docker-compose.yml and added another network to the xmpp service, so that traefik can also access the ports and not just the other container within the application network, because I got this: This is fixed now and it feels that I move constantly forward, but still have some issues.

klangborste commented 2 years ago

wss://domain.xyz/colibri-ws/192.168.170.5/33341874f690527/ccd1f9cd?pwd=somerandomkeywic8zh3h3qend I see this in the console. I guess the internal IP address does not belong there ;-) Do you have maybe a hint?

saghul commented 2 years ago

The private IP is ok, that should be the JVB container IP, is it not?

klangborste commented 2 years ago

This is correct, but should that be published in the internet through the reverse proxy?

saghul commented 2 years ago

Yep, since the reverse proxy will extract it and direct the traffic there.

klangborste commented 2 years ago

Ok thanks, I investigate a little further and filtered out all error from the dev console:

WebRTC: ICE failed, add a STUN server and see about:webrtc for more details

<3924/ti.prototype.errorCallback>: Get stats error DOMException: RTCPeerConnection is gone (did you enter Offline mode?)

UnhandledError: RTCPeerConnection is gone (did you enter Offline mode?) Script: null Line: null Column: null StackTrace: 7733/k/e.RTCPeerConnection.prototype.getStats@h

That are the three main errors. I will check now docker logs if anything happens in follow mode when I try to play around with jitsi.

saghul commented 2 years ago

WebRTC: ICE failed, add a STUN server and see about:webrtc for more details

This suggests your port 10000 UDP is not reachable / properly forwarded to the JVB container.

klangborste commented 2 years ago

I had thousand ideas, tried them out but really none helped so far, now I puted also all container into the traefik2 network so that every container and port could be used cross-wise, to exclude docker network problems. Did I need to forward the TCP port for JVB also?

I still get errors in the console for "wss://meet.domain.xyz/colibri-ws/192.168.170.5/6a45590466f95770/d95e036b?pwd=gf6f8cq1ga1unr5ts5flcg0oe" and also: "WebRTC: ICE failed, add a STUN server and see about:webrtc for more details".

My config now: Entry Point: - --entryPoints.video.address=:10000/udp

docker-compose.yml-snippet

  # Video bridge
  jvb:
      container_name: jitsi-jvb
      image: jitsi/jvb
      restart: unless-stopped
      expose:
          - "10000/udp"
      volumes:
          - ${DOCKERDIR}/jitsi/jvb:/config
      environment:
          - DOCKER_HOST_ADDRESS
          - ENABLE_COLIBRI_WEBSOCKET
          - ENABLE_OCTO
          - JVB_AUTH_USER
          - JVB_AUTH_PASSWORD
          - JVB_BREWERY_MUC
          - JVB_PORT
          - JVB_MUC_NICKNAME
          - JVB_STUN_SERVERS
          - JVB_OCTO_BIND_ADDRESS
          - JVB_OCTO_PUBLIC_ADDRESS
          - JVB_OCTO_BIND_PORT
          - JVB_OCTO_REGION
          - JVB_WS_DOMAIN
          - JVB_WS_SERVER_ID
          - PUBLIC_URL
          - SENTRY_DSN="${JVB_SENTRY_DSN:-0}"
          - SENTRY_ENVIRONMENT
          - SENTRY_RELEASE
          - COLIBRI_REST_ENABLED
          - SHUTDOWN_REST_ENABLED
          - TZ
          - XMPP_AUTH_DOMAIN
          - XMPP_INTERNAL_MUC_DOMAIN
          - XMPP_SERVER
      depends_on:
          - prosody
      networks:
        # jitsi_net:
          t2_proxy:
      labels:
          - "traefik.enable=true"
          ## HTTP Routers
          - "traefik.http.routers.jitsi-jvb.entrypoints=video"
          - "traefik.http.routers.jitsi-jvb.rule=Host(`meet.$DOMAINNAME`) && PathPrefix(`/colibri-ws`)"
          - "traefik.http.routers.jitsi-jvb.service=jitsi-jvb"
          ## Middlewares
          - "traefik.http.routers.jitsi-jvb.middlewares=chain-no-auth@file"
          ## HTTP Services
          - "traefik.http.services.jitsi-jvb.loadbalancer.server.port=10000"
          - "traefik.docker.network=t2_proxy"

saghul commented 2 years ago

Sorry, that's as far as I go, since I'm not familiar with Traefik.

prayagsingh commented 2 years ago

Few pointers that I can suggest.

Don't enable Traefik for the JVB.
Use Traefik with a web container only.
Create two different external networks (not necessary but good for debugging ) for example proxy for Traefik and jitsi for jitsi setup. Traefik will be in the proxy network and service web will be present in both the proxy and jitsi network. Other jitsi components will be in the jitsi network.

I'm using the same network config with my deployment using docker-swarm. And also as @saghul mentioned, you have to ensure that the port 10000 UDP is accessible from outside.

klangborste commented 2 years ago

Thanks @saghul for your help so far!

@prayagsingh good hints.

Is it not possible to use JVB with websockets via the reverse proxy? Do I need to bypass the traefik reverse proxy and use the port from the docker container to the host and then to the external world?
You mean traefik should be used only with jitsi/web?
Thats what I already did in my last code snippet you see still an commented second network, thats what I did before, but I thought that this caused some error because the web console from the browser after a connect showed me the internal IP in a jvb call that directed to the IP from the jitsi app network and not the traefik proxy network

I did open my firewall for UDP 10000 to the internet.

prayagsingh commented 2 years ago

Is it not possible to use JVB with websockets via the reverse proxy

I never tried it so not sure. But there are some threads in the community that mentioned that the passing jvb traffic through traefik causing some issues.

Do I need to bypass the traefik reverse proxy and use the port from the docker container to the host and then to the external world? yes

You mean traefik should be used only with jitsi/web? yes

3. Thats what I already did in my last code snippet you see still an commented second network, thats what I did before, but I thought that this caused some error because the web console from the browser after a connect showed me the internal IP in a jvb call that directed to the IP from the jitsi app network and not the traefik proxy network

Can you try https://your_domain/xmpp-websocket in the browser and check if it returns this It works! Now point your WebSocket client to this URL to connect to Prosody.? If it doesn't then prosody is unreachable.

Also if you are enabling traefik in the jvb then you have to use the Traefik UDP router.

klangborste commented 2 years ago

I think I got it now! 🎉 It works fine with JVB over Traefik 2 and was tested with 5 users. I just pulled the new version of "docker-jitsi-meet" from GitHub (stable-7210-2, had just the stable-7210 before). After that read through some threads again and decided to start from scratch. I took the base from the "docker-compose.yml" and the ".env" from the tar.gz and just edited what is really needed for my setup, after the first success I tried to get more and more to a minimum setup of exposed ports.

I am happy now, but maybe somebody can check my last question before I am too euphoric :-)

Downloaded the https://github.com/jitsi/docker-jitsi-meet/releases/latest
Untar the content from the tarball
cp env.example .env
./gen-passwords.sh
mkdir -p jitsi/{web/crontabs,web/letsencrypt,transcripts,prosody/config,prosody/prosody-plugins-custom,jicofo,jvb,jigasi,jibri}
Edit the .env variables. The important one are for me:
- CONFIG=
- PUBLIC_URL=
- DOCKER_HOST_ADDRESS= (Should be the Docker host ip if your are behind NAT)

Edit or create the docker-compose.yml Important parts:

entryPoints for traefik - --entryPoints.video.address=:10000/udp

ports published from traefik

ports:        
- target: 10000
published: 10000
protocol: udp
mode: host

traefik labels on jitsi-web

labels:
  - "traefik.enable=true"
  ## HTTP Routers
  - "traefik.http.routers.jitsi-web.entrypoints=https"
  - "traefik.http.routers.jitsi-web.rule=Host(`meet.$DOMAINNAME`)"
  - "traefik.http.routers.jitsi-web.tls=true"
  - "traefik.http.routers.jitsi-web.service=jitsi-web"
  ## Middlewares
  - "traefik.http.routers.jitsi-web.middlewares=chain-no-auth@file"
  ## HTTP Services
  - "traefik.http.services.jitsi-web.loadbalancer.server.port=80"
  - "traefik.docker.network=t2_proxy"

traefik labels on jvb / jitsi video bridge

labels:
  - "traefik.enable=true"
  ## HTTP Routers
  - "traefik.udp.routers.jvb.entrypoints=video"
  - "traefik.udp.routers.jvb.service=jvb"
  ## HTTP Services
  - "traefik.udp.services.jvb.loadbalancer.server.port=10000"
  - "traefik.docker.network=t2_proxy"

udp routers/services must be used as @prayagsingh mentioned in the (https://doc.traefik.io/traefik/routing/providers/docker/#udp) article

"Snipped" from the services part:

Yes its extreme long :-D but I wanted to be super clear if somebody has the same issue as me!

  ##~~## Jitsi Video Conferencing ##~~##
  # Frontend
  web:
      container_name: jitsi-web
      image: jitsi/web:latest
      restart: ${RESTART_POLICY}
      expose:
          - '80'
      volumes:
          - ${DOCKERDIR}/jitsi/web:/config:Z
          - ${DOCKERDIR}/jitsi/web/crontabs:/var/spool/cron/crontabs:Z
          - ${DOCKERDIR}/jitsi/transcripts:/usr/share/jitsi-meet/transcripts:Z
      environment:
          - AMPLITUDE_ID
          - ANALYTICS_SCRIPT_URLS
          - ANALYTICS_WHITELISTED_EVENTS
          - CALLSTATS_CUSTOM_SCRIPT_URL
          - CALLSTATS_ID
          - CALLSTATS_SECRET
          - CHROME_EXTENSION_BANNER_JSON
          - CONFCODE_URL
          - CONFIG_EXTERNAL_CONNECT
          - DEFAULT_LANGUAGE
          - DEPLOYMENTINFO_ENVIRONMENT
          - DEPLOYMENTINFO_ENVIRONMENT_TYPE
          - DEPLOYMENTINFO_REGION
          - DEPLOYMENTINFO_SHARD
          - DEPLOYMENTINFO_USERREGION
          - DESKTOP_SHARING_FRAMERATE_MIN
          - DESKTOP_SHARING_FRAMERATE_MAX
          - DIALIN_NUMBERS_URL
          - DIALOUT_AUTH_URL
          - DIALOUT_CODES_URL
          - DISABLE_AUDIO_LEVELS
          - DISABLE_DEEP_LINKING
          - DISABLE_GRANT_MODERATOR
          - DISABLE_HTTPS
          - DISABLE_KICKOUT
          - DISABLE_POLLS
          - DISABLE_REACTIONS
          - DROPBOX_APPKEY
          - DROPBOX_REDIRECT_URI
          - DYNAMIC_BRANDING_URL
          - ENABLE_AUDIO_PROCESSING
          - ENABLE_AUTH
          - ENABLE_BREAKOUT_ROOMS
          - ENABLE_CALENDAR
          - ENABLE_COLIBRI_WEBSOCKET
          - ENABLE_FILE_RECORDING_SERVICE
          - ENABLE_FILE_RECORDING_SERVICE_SHARING
          - ENABLE_FLOC
          - ENABLE_GUESTS
          - ENABLE_HSTS
          - ENABLE_HTTP_REDIRECT
          - ENABLE_IPV6
          - ENABLE_LETSENCRYPT
          - ENABLE_LIPSYNC
          - ENABLE_NO_AUDIO_DETECTION
          - ENABLE_NOISY_MIC_DETECTION
          - ENABLE_PREJOIN_PAGE
          - ENABLE_P2P
          - ENABLE_WELCOME_PAGE
          - ENABLE_CLOSE_PAGE
          - ENABLE_RECORDING
          - ENABLE_REMB
          - ENABLE_REQUIRE_DISPLAY_NAME
          - ENABLE_SIMULCAST
          - ENABLE_STATS_ID
          - ENABLE_STEREO
          - ENABLE_SUBDOMAINS
          - ENABLE_TALK_WHILE_MUTED
          - ENABLE_TCC
          - ENABLE_TRANSCRIPTIONS
          - ENABLE_XMPP_WEBSOCKET
          - ETHERPAD_PUBLIC_URL
          - ETHERPAD_URL_BASE
          - GOOGLE_ANALYTICS_ID
          - GOOGLE_API_APP_CLIENT_ID
          - HIDE_PREMEETING_BUTTONS
          - INVITE_SERVICE_URL
          - JICOFO_AUTH_USER
          - LETSENCRYPT_DOMAIN
          - LETSENCRYPT_EMAIL
          - LETSENCRYPT_USE_STAGING
          - MATOMO_ENDPOINT
          - MATOMO_SITE_ID
          - MICROSOFT_API_APP_CLIENT_ID
          - NGINX_RESOLVER
          - NGINX_WORKER_PROCESSES
          - NGINX_WORKER_CONNECTIONS
          - PEOPLE_SEARCH_URL
          - PUBLIC_URL
          - P2P_PREFERRED_CODEC
          - RESOLUTION
          - RESOLUTION_MIN
          - RESOLUTION_WIDTH
          - RESOLUTION_WIDTH_MIN
          - START_AUDIO_MUTED
          - START_AUDIO_ONLY
          - START_BITRATE
          - START_SILENT
          - START_WITH_AUDIO_MUTED
          - START_VIDEO_MUTED
          - START_WITH_VIDEO_MUTED
          - TESTING_CAP_SCREENSHARE_BITRATE
          - TESTING_OCTO_PROBABILITY
          - TOKEN_AUTH_URL
          - TOOLBAR_BUTTONS
          - TZ
          - VIDEOQUALITY_BITRATE_H264_LOW
          - VIDEOQUALITY_BITRATE_H264_STANDARD
          - VIDEOQUALITY_BITRATE_H264_HIGH
          - VIDEOQUALITY_BITRATE_VP8_LOW
          - VIDEOQUALITY_BITRATE_VP8_STANDARD
          - VIDEOQUALITY_BITRATE_VP8_HIGH
          - VIDEOQUALITY_BITRATE_VP9_LOW
          - VIDEOQUALITY_BITRATE_VP9_STANDARD
          - VIDEOQUALITY_BITRATE_VP9_HIGH
          - VIDEOQUALITY_ENFORCE_PREFERRED_CODEC
          - VIDEOQUALITY_PREFERRED_CODEC
          - XMPP_AUTH_DOMAIN
          - XMPP_BOSH_URL_BASE
          - XMPP_DOMAIN
          - XMPP_GUEST_DOMAIN
          - XMPP_MUC_DOMAIN
          - XMPP_RECORDER_DOMAIN
      networks:
          jitsi_net:
          t2_proxy:
      labels:
          - "traefik.enable=true"
          ## HTTP Routers
          - "traefik.http.routers.jitsi-web.entrypoints=https"
          - "traefik.http.routers.jitsi-web.rule=Host(`meet.$DOMAINNAME`)"
          - "traefik.http.routers.jitsi-web.tls=true"
          - "traefik.http.routers.jitsi-web.service=jitsi-web"
          ## Middlewares
          - "traefik.http.routers.jitsi-web.middlewares=chain-no-auth@file"
          ## HTTP Services
          - "traefik.http.services.jitsi-web.loadbalancer.server.port=80"
          - "traefik.docker.network=t2_proxy"

  # XMPP server
  prosody:
      container_name: jitsi-xmpp
      image: jitsi/prosody:latest
      restart: ${RESTART_POLICY}
      expose:
          - '5222'
          - '5347'
          - '5280'
      volumes:
          - ${DOCKERDIR}/jitsi/prosody/config:/config:Z
          - ${DOCKERDIR}/jitsi/prosody/prosody-plugins-custom:/prosody-plugins-custom:Z
      environment:
          - AUTH_TYPE
          - DISABLE_POLLS
          - ENABLE_AUTH
          - ENABLE_AV_MODERATION
          - ENABLE_BREAKOUT_ROOMS
          - ENABLE_GUESTS
          - ENABLE_LOBBY
          - ENABLE_XMPP_WEBSOCKET
          - GLOBAL_CONFIG
          - GLOBAL_MODULES
          - JIBRI_RECORDER_USER
          - JIBRI_RECORDER_PASSWORD
          - JIBRI_XMPP_USER
          - JIBRI_XMPP_PASSWORD
          - JICOFO_AUTH_USER
          - JICOFO_AUTH_PASSWORD
          - JICOFO_COMPONENT_SECRET
          - JIGASI_XMPP_USER
          - JIGASI_XMPP_PASSWORD
          - JVB_AUTH_USER
          - JVB_AUTH_PASSWORD
          - JWT_APP_ID
          - JWT_APP_SECRET
          - JWT_ACCEPTED_ISSUERS
          - JWT_ACCEPTED_AUDIENCES
          - JWT_ASAP_KEYSERVER
          - JWT_ALLOW_EMPTY
          - JWT_AUTH_TYPE
          - JWT_TOKEN_AUTH_MODULE
          - MATRIX_UVS_URL
          - MATRIX_UVS_ISSUER
          - MATRIX_UVS_AUTH_TOKEN
          - MATRIX_UVS_SYNC_POWER_LEVELS
          - LOG_LEVEL
          - LDAP_AUTH_METHOD
          - LDAP_BASE
          - LDAP_BINDDN
          - LDAP_BINDPW
          - LDAP_FILTER
          - LDAP_VERSION
          - LDAP_TLS_CIPHERS
          - LDAP_TLS_CHECK_PEER
          - LDAP_TLS_CACERT_FILE
          - LDAP_TLS_CACERT_DIR
          - LDAP_START_TLS
          - LDAP_URL
          - LDAP_USE_TLS
          - PUBLIC_URL
          - TURN_CREDENTIALS
          - TURN_HOST
          - TURNS_HOST
          - TURN_PORT
          - TURNS_PORT
          - TZ
          - XMPP_DOMAIN
          - XMPP_AUTH_DOMAIN
          - XMPP_GUEST_DOMAIN
          - XMPP_MUC_DOMAIN
          - XMPP_INTERNAL_MUC_DOMAIN
          - XMPP_MODULES
          - XMPP_MUC_MODULES
          - XMPP_INTERNAL_MUC_MODULES
          - XMPP_RECORDER_DOMAIN
      networks:
          jitsi_net:
              aliases:
                  - ${XMPP_SERVER}

  # Focus component
  jicofo:
      container_name: jitsi-jicofo
      image: jitsi/jicofo:latest
      restart: ${RESTART_POLICY}
      volumes:
          - ${DOCKERDIR}/jitsi/jicofo:/config:Z
      environment:
          - AUTH_TYPE
          - BRIDGE_AVG_PARTICIPANT_STRESS
          - BRIDGE_STRESS_THRESHOLD
          - ENABLE_AUTH
          - ENABLE_AUTO_OWNER
          - ENABLE_CODEC_VP8
          - ENABLE_CODEC_VP9
          - ENABLE_CODEC_H264
          - ENABLE_OCTO
          - ENABLE_RECORDING
          - ENABLE_SCTP
          - ENABLE_AUTO_LOGIN
          - JICOFO_AUTH_USER
          - JICOFO_AUTH_PASSWORD
          - JICOFO_ENABLE_BRIDGE_HEALTH_CHECKS
          - JICOFO_CONF_INITIAL_PARTICIPANT_WAIT_TIMEOUT
          - JICOFO_CONF_SINGLE_PARTICIPANT_TIMEOUT
          - JICOFO_ENABLE_HEALTH_CHECKS
          - JICOFO_SHORT_ID
          - JICOFO_RESERVATION_ENABLED
          - JICOFO_RESERVATION_REST_BASE_URL
          - JIBRI_BREWERY_MUC
          - JIBRI_REQUEST_RETRIES
          - JIBRI_PENDING_TIMEOUT
          - JIGASI_BREWERY_MUC
          - JIGASI_SIP_URI
          - JVB_BREWERY_MUC
          - MAX_BRIDGE_PARTICIPANTS
          - OCTO_BRIDGE_SELECTION_STRATEGY
          - SENTRY_DSN="${JICOFO_SENTRY_DSN:-0}"
          - SENTRY_ENVIRONMENT
          - SENTRY_RELEASE
          - TZ
          - XMPP_DOMAIN
          - XMPP_AUTH_DOMAIN
          - XMPP_INTERNAL_MUC_DOMAIN
          - XMPP_MUC_DOMAIN
          - XMPP_RECORDER_DOMAIN
          - XMPP_SERVER
      depends_on:
          - prosody
      networks:
          jitsi_net:

  # Video bridge
  jvb:
      container_name: jitsi-jvb
      image: jitsi/jvb:latest
      restart: ${RESTART_POLICY}
      expose:
          - '10000/udp'
      volumes:
          - ${DOCKERDIR}/jitsi/jvb:/config:Z
      environment:
          - DOCKER_HOST_ADDRESS
          - ENABLE_COLIBRI_WEBSOCKET
          - ENABLE_OCTO
          - JVB_AUTH_USER
          - JVB_AUTH_PASSWORD
          - JVB_BREWERY_MUC
          - JVB_PORT
          - JVB_MUC_NICKNAME
          - JVB_STUN_SERVERS
          - JVB_OCTO_BIND_ADDRESS
          - JVB_OCTO_PUBLIC_ADDRESS
          - JVB_OCTO_BIND_PORT
          - JVB_OCTO_REGION
          - JVB_WS_DOMAIN
          - JVB_WS_SERVER_ID
          - PUBLIC_URL
          - SENTRY_DSN="${JVB_SENTRY_DSN:-0}"
          - SENTRY_ENVIRONMENT
          - SENTRY_RELEASE
          - COLIBRI_REST_ENABLED
          - SHUTDOWN_REST_ENABLED
          - TZ
          - XMPP_AUTH_DOMAIN
          - XMPP_INTERNAL_MUC_DOMAIN
          - XMPP_SERVER
      depends_on:
          - prosody
      networks:
          jitsi_net:
          t2_proxy:
      labels:
          - "traefik.enable=true"
          ## HTTP Routers
          - "traefik.udp.routers.jvb.entrypoints=video"
          - "traefik.udp.routers.jvb.service=jvb"
          ## HTTP Services
          - "traefik.udp.services.jvb.loadbalancer.server.port=10000"
          - "traefik.docker.network=t2_proxy"

Thanks for all that helped!

I have one last question. The 10000/UDP port must be available from the web browsers/web clients for jitsi, so if you expose your server to the internet the udp port must be externally reachable?

Hope this helps maybe also others.

klangborste commented 2 years ago

Is that infrastructure right? I added just Traefik where it needs to be?

The jigasi part can be ignored in my case, dont know if this is also traefik routeable

saghul commented 2 years ago

It's Traefik processing the UDP traffic?

klangborste commented 2 years ago

I would say yes :-)

1433213db73f   jitsi/jicofo:latest             "/init"                  9 hours ago   Up 9 hours                                                                                                                                       jitsi-jicofo
ecf46c55b707   jitsi/jvb:latest                "/init"                  9 hours ago   Up 9 hours             10000/udp                                                                                                                 jitsi-jvb
ad41881ac3cc   jitsi/prosody:latest            "/init"                  9 hours ago   Up 9 hours             5222/tcp, 5280/tcp, 5347/tcp                                                                                              jitsi-xmpp
16e18be1d6b0   jitsi/web:latest                "/init"                  9 hours ago   Up 9 hours             80/tcp, 443/tcp                                                                                                           jitsi-web
5419b1a57f38   traefik:latest                  "/entrypoint.sh --gl…"   9 hours ago   Up 9 hours             0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:10000->10000/udp, :::10000->10000/udp   traefik

saghul commented 2 years ago

Cool! Should we close this now then?

klangborste commented 2 years ago

Is it necessary that 10000/udp port must be reachable from the client or is possible that it got tunneled from the proxy with another protocol?

saghul commented 2 years ago

It must be reachable by the client browsers yeah. Generally there is no benefit in proxying.

klangborste commented 2 years ago

Ok then thanks a lot again. Then everyting is working as it should and my question to understand it better are answered.

jitsi / docker-jitsi-meet

WebSocket Error with Traefik 2 #1271