Connection closes soon after an establishing RTSP connection

jattind commented 1 year ago

Which version are you using?

v0.21.6

Which operating system are you using?

[ ] Linux amd64 standard
[x ] Linux amd64 Docker
[ ] Linux arm64 standard
[ ] Linux arm64 Docker
[ ] Linux arm7 standard
[ ] Linux arm7 Docker
[ ] Linux arm6 standard
[ ] Linux arm6 Docker
[ ] Windows amd64 standard
[ ] Windows amd64 Docker (WSL backend)
[ ] macOS amd64 standard
[ ] macOS amd64 Docker
[ ] Other (please describe)

Describe the issue

Deployed rtsp-simple-server in a kubernetes cluster. If I use the pod IP to send a stream, the connection remains established, but if I create a service for the pod and try to connect a client, the connection gets established, but immediately terminates.

Describe how to replicate the issue

Deploy the server using a yaml file as shown bellow kubectl apply -f rtsp-server-deployment.yaml kubectl apply -f rtsp-service.yaml try connecting an rtsp stream to the rts-simple-server View the logs for the POD kubectl logs -n bare srtp-pod

rtsp-server-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rtsp-server
  namespace: bare
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      io.kompose.service: rtsp-server
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        io.kompose.network/edge-bare-edge-network: "true"
        io.kompose.service: rtsp-server
    spec:
      containers:
      - image: harbor10165.senode.dev/sgs/simple-rtsp-server
        imagePullPolicy: IfNotPresent
        name: rtsp-server
        ports:
        - containerPort: 8554
          protocol: TCP
        resources:
          limits:
            cpu: "1"
            memory: "1073741824"
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

rtsp-service.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: rtsp-service-tcp
  namespace: bare
spec:
  selector:
    io.kompose.service: rtsp-server
  ports:
    - name: rtsp-tcp
      protocol: TCP
      port: 8554
      targetPort: 8554
  type: ClusterIP

Did you attach the server logs?

yes

Working
=======
kubectl logs -f -n bare rtsp-server-6f994c88bb-bzm29
2023/05/31 20:13:35 INF MediaMTX / rtsp-simple-server v0.21.6
2023/05/31 20:13:35 INF [RTSP] listener opened on :8554 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP)
2023/05/31 20:13:35 INF [RTMP] listener opened on :1935
2023/05/31 20:13:35 INF [HLS] listener opened on :8888
2023/05/31 20:13:35 INF [WebRTC] listener opened on :8889 (HTTP)

- deploy infrence1-service
2023/05/31 20:21:01 INF [RTSP] [conn 10.42.172.91:59548] opened
2023/05/31 20:21:01 INF [RTSP] [session 965fa869] created by 10.42.172.91:59548
2023/05/31 20:21:02 INF [RTSP] [session 965fa869] is publishing to path 'inference_1', with UDP, 1 track (H264)

- deploy inference2-service
2023/05/31 20:21:50 INF [RTSP] [conn 10.42.172.94:51702] opened
2023/05/31 20:21:50 INF [RTSP] [session a1e73eee] created by 10.42.172.94:51702
2023/05/31 20:21:51 INF [RTSP] [session a1e73eee] is publishing to path 'inference_2', with UDP, 1 track (H264)

- deploy stream-ui
2023/05/31 20:22:36 INF [RTSP] [conn 10.42.172.95:33826] opened
2023/05/31 20:22:36 INF [RTSP] [session 5b646232] created by 10.42.172.95:33826
2023/05/31 20:22:36 INF [RTSP] [session 5b646232] is reading from path 'inference_1', with TCP, 1 track (H264)
2023/05/31 20:22:42 INF [RTSP] [conn 10.42.172.95:56028] opened
2023/05/31 20:22:42 INF [RTSP] [session bac3c23e] created by 10.42.172.95:56028
2023/05/31 20:22:42 INF [RTSP] [session bac3c23e] is reading from path 'inference_2', with TCP, 1 track (H264)

Failing
========

kubectl logs -f -n bare rtsp-server-6f994c88bb-8r7jf
2023/05/31 20:00:01 INF MediaMTX / rtsp-simple-server v0.21.6
2023/05/31 20:00:01 INF [RTSP] listener opened on :8554 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP)
2023/05/31 20:00:01 INF [RTMP] listener opened on :1935
2023/05/31 20:00:01 INF [HLS] listener opened on :8888
2023/05/31 20:00:01 INF [WebRTC] listener opened on :8889 (HTTP)

- deploy inference1-service
2023/05/31 20:01:12 INF [RTSP] [conn 10.42.172.84:50090] opened
2023/05/31 20:01:12 INF [RTSP] [session 0788142a] created by 10.42.172.84:50090
2023/05/31 20:01:13 INF [RTSP] [session 0788142a] is publishing to path 'inference_1', with UDP, 1 track (H264)
2023/05/31 20:01:23 INF [RTSP] [conn 10.42.172.84:50090] closed (terminated)
2023/05/31 20:01:23 INF [RTSP] [session 0788142a] destroyed (session timed out)
2023/05/31 20:01:23 INF [RTSP] [conn 10.42.172.84:59328] opened

- deployed inference1-service
2023/05/31 20:03:08 INF [RTSP] [conn 10.42.172.86:56622] opened
2023/05/31 20:03:08 INF [RTSP] [session 5e640c7a] created by 10.42.172.86:56622
2023/05/31 20:03:09 INF [RTSP] [session 5e640c7a] is publishing to path 'inference_2', with UDP, 1 track (H264)
2023/05/31 20:03:19 INF [RTSP] [conn 10.42.172.86:56622] closed (terminated)
2023/05/31 20:03:19 INF [RTSP] [session 5e640c7a] destroyed (session timed out)
2023/05/31 20:03:19 INF [RTSP] [conn 10.42.172.86:44406] opened
2023/05/31 20:04:16 INF [RTSP] [conn 10.42.172.87:32798] opened
2023/05/31 20:04:16 INF [RTSP] [conn 10.42.172.87:32798] closed (no one is publishing to path 'inference_1')
2023/05/31 20:04:16 INF [RTSP] [conn 10.42.172.87:32804] opened
2023/05/31 20:04:16 INF [RTSP] [conn 10.42.172.87:32804] closed (no one is publishing to path 'inference_2')
2023/05/31 20:04:21 INF [RTSP] [conn 10.42.172.87:59742] opened
2023/05/31 20:04:21 INF [RTSP] [conn 10.42.172.87:59742] closed (no one is publishing to path 'inference_1')
2023/05/31 20:04:21 INF [RTSP] [conn 10.42.172.87:59746] opened
2023/05/31 20:04:21 INF [RTSP] [conn 10.42.172.87:59746] closed (no one is publishing to path 'inference_2')

Did you attach a network dump?

no

aler9 commented 1 year ago

Hello, RTSP has multiple subprotocols, and the default one (UDP) doesn't work when there's a NAT/firewall between server and clients. A Kubernetes service acts as a NAT between the server pod and the clients, since it has a dedicated IP.

Set TCP as the default transport protocol:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rtsp-server
  namespace: bare
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      io.kompose.service: rtsp-server
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        io.kompose.network/edge-bare-edge-network: "true"
        io.kompose.service: rtsp-server
    spec:
      containers:
      - image: harbor10165.senode.dev/sgs/simple-rtsp-server
        imagePullPolicy: IfNotPresent
        name: rtsp-server
        ports:
        - containerPort: 8554
          protocol: TCP
        resources:
          limits:
            cpu: "1"
            memory: "1073741824"
        env:
        - name: MTX_PROTOCOLS
          value: tcp
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

Alternatively you can replace the Deployment with a StatefulSet, make the service Headless and publish to rtsp-server-0.rtsp-service-tcp. In this way, the service doesn't perform any load balancing, doesn't act as NAT and you can use UDP.

jattind commented 1 year ago

I tried forcing TCP connection only as per above using the latest release of rtsp-simple-server. It fails to create create a connection:

kubectl logs -f -n bare rtsp-server-5898dcfd5f-8pgwl
2023/06/05 17:20:43 INF MediaMTX / rtsp-simple-server v0.23.4
2023/06/05 17:20:43 INF [RTSP] listener opened on :8554 (TCP)
2023/06/05 17:20:43 INF [RTMP] listener opened on :1935
2023/06/05 17:20:43 INF [HLS] listener opened on :8888
2023/06/05 17:20:43 INF [WebRTC] listener opened on :8889 (HTTP)

2023/06/05 17:21:47 INF [RTSP] [conn 10.42.172.122:45392] opened
2023/06/05 17:21:47 INF [RTSP] [session 7f83351b] created by 10.42.172.122:45392
2023/06/05 17:21:47 INF [RTSP] [conn 10.42.172.122:45392] closed (EOF)
2023/06/05 17:21:47 INF [RTSP] [session 7f83351b] destroyed (not in use)

aler9 commented 1 year ago

In order to debug what's happening, please post logs of the TCP-based connection with the parameter logLevel: debug.

github-actions[bot] commented 6 months ago

This issue is being locked automatically because it has been closed for more than 6 months. Please open a new issue in case you encounter a similar problem.

bluenviron / mediamtx