OBS segfaulting when attempting to connect to RSVP server

[WhyDoWeWonder]

When using the added menu under Tools to open up the RSVP server window and clicking "start" with the correct port, obs-studio segfaults.

Things I've tried:

• multiple different RSVP server implementations, ffmpeg can play video streams to them fine so the issue seems to lie with obs-studio or obs-rtspserver.
• building the plugin from source and using the files extracted from the prebuilt AUR package
• trying versions obs-rtspserver versions 2.0.1, 2.0.0, and 1.4.1.

I am on Linux and the stacktrace from debugging with gdb on a debug build is as follows

info: [NVENC encoder: 'rtsp_output_video'] settings:
        rate_control: CBR
        bitrate:      2500
        cqp:          0
        keyint:       250
        preset:       hq
        profile:      high
        width:        1920
        height:       1040
        2-pass:       false
        b-frames:     4
        GPU:          0

[New Thread 0x7fff62919640 (LWP 268420)]
[New Thread 0x7fff62118640 (LWP 268421)]
[New Thread 0x7fff61917640 (LWP 268422)]
info: libfdk_aac encoder created
info: libfdk_aac bitrate: 128, channels: 2
--Type <RET> for more, q to quit, c to continue without paging--

Thread 1 "obs" received signal SIGSEGV, Segmentation fault.
ff_avc_find_startcode_internal (end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, p=0x0)
    at /usr/src/debug/obs-studio-26.1.2/libobs/obs-avc.c:57
warning: Source file is more recent than executable.
57                      if (p[0] == 0 && p[1] == 0 && p[2] == 1)```

If someone could shed some light on what's going on here that would be greatly appreciated, I have no clue what's going on.

[brodi1]

Hi @WhyDoWeWonder Does everything work for you if switch to software x264 encoding?

[iamscottxu]

info: [NVENC encoder: 'rtsp_output_video'] settings: rate_control: CBR bitrate: 2500 cqp: 0 keyint: 250 preset: hq profile: high width: 1920 height: 1040 2-pass: false b-frames: 4 GPU: 0

Using NVENC and enabled output scaling may cause the program to crash, please try to disable output scaling.