search

obsproject / obs-studio

OBS Studio - Free and open source software for live streaming and screen recording
https://obsproject.com
GNU General Public License v2.0
57.65k stars 7.74k forks source link

Suspected Memory Leak causes OBS to be Killed by oom_killer #5828

Open avrovulcanxh607 opened 2 years ago

avrovulcanxh607 commented 2 years ago

Operating System Info

Ubuntu 20.04

Other OS

No response

OBS Studio Version

27.1.3

OBS Studio Version (Other)

No response

OBS Studio Log URL

n/a

OBS Studio Crash Log URL

No response

Expected Behavior

Stable RAM Usage

Current Behavior

The RAM usage continuously climbs untill the OOM task killer wakes up and kills it (so there's nothing of interest in the OBS Logs)

Steps to Reproduce

I use OBS to stream 24/7 from a server. I've noticed that almost exactly every 24 hours it seemingly crashes, but leaves no information in it's logs. Checking syslog revealed that the out-of-memory task killer gets activated and kills it, with maybe 24 odd gigabytes of RAM usage. This is only streaming at standard definition with only a handful of scenes, so I really can't see why this would be happening. (The same setup has ran on windows without issue for weeks on end).

To aid in debugging, I've checked that the conditions are the same in multiple situations; The RAM usage still climbs whether the system is actually streaming or not, and even when my live inputs from GStreamer and Icecast are disabled.

Anything else we should know?

I have a few plugins installed (Gstreamer and OBS-Websocket) but as mentioned above I don't think they're affecting the issue.

I'm streaming via RTMP to an internal server

I'm using browser sources to show two internal webpages

I'm also using multiple text sources

WizardCM commented 2 years ago

Please provide a session log, ideally with obs launched in --verbose mode.

avrovulcanxh607 commented 2 years ago

Apologies for the copy/paste job, but some URLs need to be censored...

10:40:42.049: CPU Name: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz
10:40:42.049: CPU Speed: 3000.000MHz
10:40:42.049: Physical Cores: 8, Logical Cores: 8
10:40:42.049: Physical Memory: 31970MB Total, 11765MB Free
10:40:42.049: Kernel Version: Linux 5.11.0-46-generic
10:40:42.049: Distribution: "Ubuntu" "20.04"
10:40:42.049: Session Type: x11
10:40:42.049: Window System: X11.0, Vendor: The X.Org Foundation, Version: 1.20.13
10:40:42.049: found alternate keycode 129 for OBS_KEY_NUMPERIOD which already has keycode 91
10:40:42.049: found alternate keycode 190 for OBS_KEY_REDO which already has keycode 137
10:40:42.049: found alternate keycode 218 for OBS_KEY_PRINT which already has keycode 107
10:40:42.049: found alternate keycode 231 for OBS_KEY_CANCEL which already has keycode 136
10:40:42.049: Portable mode: false
10:40:42.077: OBS 27.1.3 (linux)
10:40:42.077: ---------------------------------
10:40:42.124: ---------------------------------
10:40:42.124: audio settings reset:
10:40:42.124:   samples per sec: 48000
10:40:42.124:   speakers:        2
10:40:42.131: ---------------------------------
10:40:42.131: Initializing OpenGL...
10:40:42.163: Loading up OpenGL on adapter Intel Mesa Intel(R) UHD Graphics 630 (CFL GT2)
10:40:42.163: OpenGL loaded successfully, version 4.6 (Core Profile) Mesa 21.0.3, shading language 4.60
10:40:42.173: ---------------------------------
10:40:42.173: video settings reset:
10:40:42.173:   base resolution:   1280x720
10:40:42.173:   output resolution: 960x540
10:40:42.173:   downscale filter:  Bicubic
10:40:42.173:   fps:               50/1
10:40:42.173:   format:            NV12
10:40:42.173:   YUV mode:          709/Partial
10:40:42.173: NV12 texture support not available
10:40:42.175: Audio monitoring device:
10:40:42.175:   name: Default
10:40:42.175:   id: default
10:40:42.175: ---------------------------------
10:40:42.175: ---------------------------------
10:40:42.176: Loading module: decklink-captions.so
10:40:42.176: Failed to load 'en-US' text for module: 'decklink-captions.so'
10:40:42.176: ---------------------------------
10:40:42.177: Loading module: decklink-ouput-ui.so
10:40:42.177: Failed to load 'en-US' text for module: 'decklink-ouput-ui.so'
10:40:42.177: ---------------------------------
10:40:42.179: Loading module: frontend-tools.so
10:40:42.225: ---------------------------------
10:40:42.226: Loading module: image-source.so
10:40:42.226: ---------------------------------
10:40:42.233: Required module function 'obs_module_load' in module '/usr//lib/obs-plugins/libcef.so' not found, loading of module failed
10:40:42.233: Failed to load module file '/usr//lib/obs-plugins/libcef.so': -3
10:40:42.233: ---------------------------------
10:40:42.234: Required module function 'obs_module_load' in module '/usr//lib/obs-plugins/libEGL.so' not found, loading of module failed
10:40:42.234: Failed to load module file '/usr//lib/obs-plugins/libEGL.so': -3
10:40:42.234: ---------------------------------
10:40:42.240: Required module function 'obs_module_load' in module '/usr//lib/obs-plugins/libGLESv2.so' not found, loading of module failed
10:40:42.240: Failed to load module file '/usr//lib/obs-plugins/libGLESv2.so': -3
10:40:42.240: ---------------------------------
10:40:42.241: Loading module: linux-alsa.so
10:40:42.241: Failed to load 'en-GB' text for module: 'linux-alsa.so'
10:40:42.241: ---------------------------------
10:40:42.242: Loading module: linux-capture.so
10:40:42.242: Failed to load 'en-GB' text for module: 'linux-capture.so'
10:40:42.242: ---------------------------------
10:40:42.243: Loading module: linux-decklink.so
10:40:42.243: A DeckLink iterator could not be created.  The DeckLink drivers may not be installed
10:40:42.243: No blackmagic support
10:40:42.243: Failed to start search for DeckLink devices
10:40:42.243: ---------------------------------
10:40:42.244: Loading module: linux-jack.so
10:40:42.244: Failed to load 'en-GB' text for module: 'linux-jack.so'
10:40:42.244: ---------------------------------
10:40:42.244: Loading module: linux-pulseaudio.so
10:40:42.244: Failed to load 'en-GB' text for module: 'linux-pulseaudio.so'
10:40:42.244: ---------------------------------
10:40:42.245: Loading module: linux-v4l2.so
10:40:42.247: ---------------------------------
10:40:42.249: Loading module: obs-browser.so
10:40:42.249: Failed to load 'en-GB' text for module: 'obs-browser.so'
10:40:42.249: [obs-browser]: Version 2.16.3
10:40:42.249: [obs-browser]: CEF Version 87.1.12+g03f9336+chromium-87.0.4280.88
10:40:42.249: ---------------------------------
10:40:42.262: Loading module: obs-ffmpeg.so
10:40:42.262: os_dlopen(libnvidia-encode.so.1->libnvidia-encode.so.1): libnvidia-encode.so.1: cannot open shared object file: No such file or directory
10:40:42.262: 
10:40:42.262: FFMPEG VAAPI supported
10:40:42.262: ---------------------------------
10:40:42.263: Loading module: obs-filters.so
10:40:42.263: ---------------------------------
10:40:42.264: Loading module: obs-libfdk.so
10:40:42.264: Failed to load 'en-GB' text for module: 'obs-libfdk.so'
10:40:42.264: ---------------------------------
10:40:42.265: Loading module: obs-outputs.so
10:40:42.265: ---------------------------------
10:40:42.266: Loading module: obs-transitions.so
10:40:42.266: ---------------------------------
10:40:42.266: Loading module: obs-vst.so
10:40:42.266: Failed to load 'en-GB' text for module: 'obs-vst.so'
10:40:42.266: ---------------------------------
10:40:42.269: Loading module: obs-websocket.so
10:40:42.270: Failed to load 'en-GB' text for module: 'obs-websocket.so'
10:40:42.270: [obs-websocket] you can haz websockets (version 4.9.1)
10:40:42.270: [obs-websocket] qt version (compile-time): 5.12.8 ; qt version (run-time): 5.12.8
10:40:42.272: [obs-websocket] module loaded!
10:40:42.272: ---------------------------------
10:40:42.272: Loading module: obs-x264.so
10:40:42.272: Failed to load 'en-GB' text for module: 'obs-x264.so'
10:40:42.272: ---------------------------------
10:40:42.273: Loading module: rtmp-services.so
10:40:42.273: Failed to load 'en-GB' text for module: 'rtmp-services.so'
10:40:42.274: ---------------------------------
10:40:42.274: Loading module: text-freetype2.so
10:40:42.274: ---------------------------------
10:40:42.275: Loading module: vlc-video.so
10:40:42.275: os_dlopen(libvlc.so.5->libvlc.so.5): libvlc.so.5: cannot open shared object file: No such file or directory
10:40:42.275: 
10:40:42.275: Couldn't find VLC installation, VLC video source disabled
10:40:42.275: ---------------------------------
10:40:42.277: Loading module: obs-gstreamer.so
10:40:42.277: obs-gstreamer build: 219a629
10:40:42.285: ---------------------------------
10:40:42.285:   Loaded Modules:
10:40:42.285:     obs-gstreamer.so
10:40:42.285:     vlc-video.so
10:40:42.285:     text-freetype2.so
10:40:42.285:     rtmp-services.so
10:40:42.285:     obs-x264.so
10:40:42.285:     obs-websocket.so
10:40:42.285:     obs-vst.so
10:40:42.285:     obs-transitions.so
10:40:42.285:     obs-outputs.so
10:40:42.285:     obs-libfdk.so
10:40:42.285:     obs-filters.so
10:40:42.285:     obs-ffmpeg.so
10:40:42.285:     obs-browser.so
10:40:42.285:     linux-v4l2.so
10:40:42.285:     linux-pulseaudio.so
10:40:42.285:     linux-jack.so
10:40:42.285:     linux-decklink.so
10:40:42.285:     linux-capture.so
10:40:42.285:     linux-alsa.so
10:40:42.285:     image-source.so
10:40:42.285:     frontend-tools.so
10:40:42.285:     decklink-ouput-ui.so
10:40:42.285:     decklink-captions.so
10:40:42.285: ---------------------------------
10:40:42.285: ==== Startup complete ===============================================
10:40:42.285: output 'virtualcam_output' (virtualcam_output) created
10:40:42.285: encoder 'simple_h264_stream' (obs_x264) created
10:40:42.285: AAC encoder bitrate mapping:
10:40:42.285:    32 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:    64 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:    96 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   128 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   160 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   192 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   224 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   256 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   288 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   320 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   352 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   384 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   416 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   448 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   480 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   512 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   544 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   576 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   608 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   640 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   672 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   704 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   736 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   768 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   800 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   832 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   864 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   896 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   928 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   960 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   992 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285:   1024 kbit/s: 'libfdk AAC Encoder' (libfdk_aac)
10:40:42.285: encoder 'simple_aac' (libfdk_aac) created
10:40:42.285: encoder 'simple_archive_aac' (libfdk_aac) created
10:40:42.285: output 'simple_file_output' (ffmpeg_muxer) created
10:40:42.285: service 'default_service' (rtmp_custom) created
10:40:42.286: obs_source_get_name: Null 'source' parameter
10:40:42.286: obs_scene_duplicate: Null 'scene' parameter
10:40:42.286: obs_transition_clear: Null 'transition' parameter
10:40:42.288: Switched to Preview/Program mode
10:40:42.288: ------------------------------------------------
10:40:42.290: All scene data cleared
10:40:42.290: ------------------------------------------------
10:40:42.290: private source 'Cut' (cut_transition) created
10:40:42.290: private source 'Fade' (fade_transition) created
10:40:42.297: source 'OS 2 Feed' (gstreamer-source) created
10:40:42.298: convert_sampler_info: 1 <= max_anisotropy <= 16 violated, selected: 0, set: 1
10:40:42.298: private source 'Scaling/Aspect Ratio' (scale_filter) created
10:40:42.298: - filter 'Scaling/Aspect Ratio' (scale_filter) added to source 'OS 2 Feed'
10:40:42.308: source 'Off Air Text' (text_ft2_source_v2) created
10:40:42.308: [Media Source 'Off Air Loop']: settings:
10:40:42.308:   input:                   <censored>/Off Air track (Loop for full effect).wav
10:40:42.308:   input_format:            (null)
10:40:42.308:   speed:                   100
10:40:42.308:   is_looping:              yes
10:40:42.308:   is_linear_alpha:         no
10:40:42.308:   is_hw_decoding:          yes
10:40:42.308:   is_clear_on_media_end:   no
10:40:42.308:   restart_on_activate:     yes
10:40:42.308:   close_when_inactive:     no
10:40:42.308: source 'Off Air Loop' (ffmpeg_source) created
10:40:42.308: [Media Source 'Bezier Loop']: settings:
10:40:42.308:   input:                   <censored>/Bezier Loop.mp4
10:40:42.308:   input_format:            (null)
10:40:42.308:   speed:                   100
10:40:42.308:   is_looping:              yes
10:40:42.308:   is_linear_alpha:         no
10:40:42.308:   is_hw_decoding:          yes
10:40:42.308:   is_clear_on_media_end:   no
10:40:42.308:   restart_on_activate:     no
10:40:42.308:   close_when_inactive:     no
10:40:42.308: source 'Bezier Loop' (ffmpeg_source) created
10:40:42.308: private source 'Colour Correction' (color_filter_v2) created
10:40:42.308: - filter 'Colour Correction' (color_filter_v2) added to source 'Bezier Loop'
10:40:42.309: source 'Off Air' (scene) created
10:40:42.309: source 'Ceefax Blue' (color_source_v3) created
10:40:42.309: [image_source: 'Ceefax Logo'] loading texture '/run/user/1000/gvfs/smb-share:server=<censored>,share=<censored>BBC_Ceefax_logo.svg.png'
10:40:42.309: Failed to open file '/run/user/1000/gvfs/smb-share:server=<censored>,share=<censored>BBC_Ceefax_logo.svg.png': No such file or directory
10:40:42.309: gs_image_file_init_internal: Failed to load file '/run/user/1000/gvfs/smb-share:server=<censored>,share=<censored>BBC_Ceefax_logo.svg.png'
10:40:42.309: [image_source: 'Ceefax Logo'] failed to load texture '/run/user/1000/gvfs/smb-share:server=<censored>,share=<censored>BBC_Ceefax_logo.svg.png'
10:40:42.309: source 'Ceefax Logo' (image_source) created
10:40:42.309: source 'Caption' (scene) created
10:40:42.309: source 'NMPTV Breakfiller' (browser_source) created
10:40:42.310: source 'Breakfiller' (scene) created
10:40:42.310: source 'NMPTV Clock' (browser_source) created
10:40:42.310: source 'Clock' (scene) created
10:40:42.310: [Media Source 'Playout 1 Source']: settings:
10:40:42.310:   input:                   /run/user/1000/gvfs/smb-share:server=<censored>,share=<censored>/Dad's Army (1971 Movie).mp4
10:40:42.310:   input_format:            (null)
10:40:42.310:   speed:                   100
10:40:42.310:   is_looping:              no
10:40:42.310:   is_linear_alpha:         no
10:40:42.310:   is_hw_decoding:          yes
10:40:42.310:   is_clear_on_media_end:   yes
10:40:42.310:   restart_on_activate:     yes
10:40:42.310:   close_when_inactive:     no
10:40:42.310: source 'Playout 1 Source' (ffmpeg_source) created
10:40:42.310: MP: Failed to open media: '/run/user/1000/gvfs/smb-share:server=<censored>,share=<censored>/Dad's Army (1971 Movie).mp4'
10:40:42.310: private source 'Compressor' (compressor_filter) created
10:40:42.310: - filter 'Compressor' (compressor_filter) added to source 'Playout 1 Source'
10:40:42.310: private source 'Limiter' (limiter_filter) created
10:40:42.310: - filter 'Limiter' (limiter_filter) added to source 'Playout 1 Source'
10:40:42.311: source 'Playout 1' (scene) created
10:40:42.311: [Media Source 'Stream Ident']: settings:
10:40:42.311:   input:                   /home/nathan/Videos/stream ident.mp4
10:40:42.311:   input_format:            (null)
10:40:42.311:   speed:                   100
10:40:42.311:   is_looping:              no
10:40:42.311:   is_linear_alpha:         no
10:40:42.311:   is_hw_decoding:          yes
10:40:42.311:   is_clear_on_media_end:   yes
10:40:42.311:   restart_on_activate:     yes
10:40:42.311:   close_when_inactive:     no
10:40:42.311: source 'Stream Ident' (ffmpeg_source) created
10:40:42.311: [Media Source 'City Ident']: settings:
10:40:42.311:   input:                   /home/nathan/Videos/city ident.mp4
10:40:42.311:   input_format:            (null)
10:40:42.311:   speed:                   100
10:40:42.311:   is_looping:              no
10:40:42.311:   is_linear_alpha:         no
10:40:42.311:   is_hw_decoding:          yes
10:40:42.311:   is_clear_on_media_end:   yes
10:40:42.311:   restart_on_activate:     yes
10:40:42.311:   close_when_inactive:     no
10:40:42.311: source 'City Ident' (ffmpeg_source) created
10:40:42.317: [Media Source 'Car Ident']: settings:
10:40:42.317:   input:                   /home/nathan/Videos/New Car Ident.mp4
10:40:42.317:   input_format:            (null)
10:40:42.317:   speed:                   100
10:40:42.317:   is_looping:              no
10:40:42.317:   is_linear_alpha:         no
10:40:42.317:   is_hw_decoding:          yes
10:40:42.317:   is_clear_on_media_end:   yes
10:40:42.317:   restart_on_activate:     yes
10:40:42.317:   close_when_inactive:     no
10:40:42.317: source 'Car Ident' (ffmpeg_source) created
10:40:42.318: source 'Ident' (scene) created
10:40:42.318: source 'OSD Coverer-Upper' (color_source_v3) created
10:40:42.318: [Media Source 'PFC Music Feed']: settings:
10:40:42.318:   input:                   http://<CENSORED_ICECAST_FEED>
10:40:42.318:   input_format:            
10:40:42.318:   speed:                   100
10:40:42.318:   is_looping:              no
10:40:42.318:   is_linear_alpha:         no
10:40:42.318:   is_hw_decoding:          yes
10:40:42.318:   is_clear_on_media_end:   no
10:40:42.318:   restart_on_activate:     no
10:40:42.318:   close_when_inactive:     no
10:40:42.318: source 'PFC Music Feed' (ffmpeg_source) created
10:40:42.319: source 'OS 2' (scene) created
10:40:42.319: [Media Source 'A T C Loop']: settings:
10:40:42.319:   input:                   /home/nathan/Music/Against the Clock Loop.wav
10:40:42.319:   input_format:            (null)
10:40:42.319:   speed:                   100
10:40:42.319:   is_looping:              yes
10:40:42.319:   is_linear_alpha:         no
10:40:42.319:   is_hw_decoding:          yes
10:40:42.319:   is_clear_on_media_end:   yes
10:40:42.319:   restart_on_activate:     yes
10:40:42.319:   close_when_inactive:     no
10:40:42.319: source 'A T C Loop' (ffmpeg_source) created
10:40:42.333: Out of space trying to render glyphs
10:40:42.335: source 'NMPTV DOG' (text_ft2_source_v2) created
10:40:42.336: private source 'Colour Correction' (color_filter_v2) created
10:40:42.336: - filter 'Colour Correction' (color_filter_v2) added to source 'NMPTV DOG'
10:40:42.337: source 'DownStream Key' (scene) created
10:40:42.337: source 'OS 1' (scene) created
10:40:42.337: source 'Black Bar' (color_source_v3) created
10:40:42.338: private source 'Colour Correction' (color_filter_v2) created
10:40:42.338: - filter 'Colour Correction' (color_filter_v2) added to source 'Black Bar'
10:40:42.351: source 'Line Ident Text' (text_ft2_source_v2) created
10:40:42.363: Out of space trying to render glyphs
10:40:42.366: source 'NMS Logo' (text_ft2_source_v2) created
10:40:42.366: [image_source: 'Colour Bars'] loading texture '/home/nathan/Pictures/bars.bmp'
10:40:42.407: source 'Colour Bars' (image_source) created
10:40:42.407: [Media Source 'Hens Loop']: settings:
10:40:42.407:   input:                   /home/nathan/Videos/hens.mp4
10:40:42.407:   input_format:            (null)
10:40:42.407:   speed:                   100
10:40:42.407:   is_looping:              yes
10:40:42.407:   is_linear_alpha:         no
10:40:42.407:   is_hw_decoding:          yes
10:40:42.407:   is_clear_on_media_end:   no
10:40:42.407:   restart_on_activate:     no
10:40:42.407:   close_when_inactive:     no
10:40:42.407: source 'Hens Loop' (ffmpeg_source) created
10:40:42.407: source 'Line Ident' (scene) created
10:40:42.415: Switched to scene 'Line Ident'
10:40:42.415: private source 'OS 2' (scene) created
10:40:42.415: private source 'OSD Coverer-Upper' (color_source_v3) created
10:40:42.415: private source 'DownStream Key' (scene) created
10:40:42.430: Out of space trying to render glyphs
10:40:42.433: private source 'NMPTV DOG' (text_ft2_source_v2) created
10:40:42.433: private source 'Colour Correction' (color_filter_v2) created
10:40:42.433: - filter 'Colour Correction' (color_filter_v2) added to source 'NMPTV DOG'
10:40:42.436: ------------------------------------------------
10:40:42.436: Loaded scenes:
10:40:42.436: - scene 'OS 1':
10:40:42.436:     - source: 'DownStream Key' (scene)
10:40:42.436: - scene 'OS 2':
10:40:42.436:     - source: 'OS 2 Feed' (gstreamer-source)
10:40:42.436:         - filter: 'Scaling/Aspect Ratio' (scale_filter)
10:40:42.436:     - source: 'PFC Music Feed' (ffmpeg_source)
10:40:42.436:     - source: 'OSD Coverer-Upper' (color_source_v3)
10:40:42.436:     - source: 'DownStream Key' (scene)
10:40:42.436: - scene 'Playout 1':
10:40:42.436:     - source: 'Playout 1 Source' (ffmpeg_source)
10:40:42.436:         - filter: 'Compressor' (compressor_filter)
10:40:42.436:         - filter: 'Limiter' (limiter_filter)
10:40:42.436:     - source: 'DownStream Key' (scene)
10:40:42.436: - scene 'Ident':
10:40:42.436:     - source: 'Car Ident' (ffmpeg_source)
10:40:42.436:     - source: 'City Ident' (ffmpeg_source)
10:40:42.436:     - source: 'Stream Ident' (ffmpeg_source)
10:40:42.436: - scene 'Clock':
10:40:42.436:     - source: 'NMPTV Clock' (browser_source)
10:40:42.436: - scene 'Breakfiller':
10:40:42.436:     - source: 'NMPTV Breakfiller' (browser_source)
10:40:42.436: - scene 'Caption':
10:40:42.436:     - source: 'Ceefax Blue' (color_source_v3)
10:40:42.436:     - source: 'Ceefax Logo' (image_source)
10:40:42.436: - scene 'Line Ident':
10:40:42.436:     - source: 'Hens Loop' (ffmpeg_source)
10:40:42.437:     - source: 'A T C Loop' (ffmpeg_source)
10:40:42.437:     - source: 'Black Bar' (color_source_v3)
10:40:42.437:         - filter: 'Colour Correction' (color_filter_v2)
10:40:42.437:     - source: 'NMS Logo' (text_ft2_source_v2)
10:40:42.437:     - source: 'Colour Bars' (image_source)
10:40:42.437:     - source: 'Line Ident Text' (text_ft2_source_v2)
10:40:42.437: - scene 'DownStream Key':
10:40:42.437:     - source: 'NMPTV DOG' (text_ft2_source_v2)
10:40:42.437:         - filter: 'Colour Correction' (color_filter_v2)
10:40:42.437: - scene 'Off Air':
10:40:42.437:     - source: 'Bezier Loop' (ffmpeg_source)
10:40:42.437:         - filter: 'Colour Correction' (color_filter_v2)
10:40:42.437:     - source: 'Off Air Loop' (ffmpeg_source)
10:40:42.437:     - source: 'Off Air Text' (text_ft2_source_v2)
10:40:42.437: ------------------------------------------------
10:40:42.514: [obs-websocket] WSServer::start: Not locked to IPv4 bindings
10:40:42.516: [obs-websocket] server started successfully on port 4444
10:40:42.516: [obs-websocket] IO thread started.
10:40:42.562: A sprite cannot be drawn without a width/height
10:40:59.303: output 'simple_stream' (rtmp_output) created
10:40:59.304: ---------------------------------
10:40:59.304: [x264 encoder: 'simple_h264_stream'] preset: veryfast
10:40:59.304: [x264 encoder: 'simple_h264_stream'] settings:
10:40:59.304:   rate_control: CBR
10:40:59.304:   bitrate:      10000
10:40:59.304:   buffer size:  10000
10:40:59.304:   crf:          23
10:40:59.304:   fps_num:      50
10:40:59.304:   fps_den:      1
10:40:59.304:   width:        960
10:40:59.304:   height:       540
10:40:59.304:   keyint:       250
10:40:59.304: 
10:40:59.306: libfdk_aac encoder created
10:40:59.306: libfdk_aac bitrate: 192, channels: 2
10:40:59.306: [rtmp stream: 'simple_stream'] Connecting to RTMP URL rtmp://192.168.1.32:1935/live/...
10:40:59.308: [rtmp stream: 'simple_stream'] Connection to rtmp://192.168.1.32:1935/live/ successful
10:40:59.311: ==== Streaming Start ===============================================
10:43:46.159: obs-browser: Uncaught SecurityError: Blocked a frame with origin "https://internal.nathanmediaservices.co.uk" from accessing a frame with origin "null".  The frame requesting access has a protocol of "https", the frame being accessed has a protocol of "chrome-error". Protocols must match.
10:43:46.159:  (source: https://internal.nathanmediaservices.co.uk/<CENSORED>:130)
ElectronicWar commented 2 years ago

Can you check after some hours of runtime if the actual OBS process has the memory leak or if it's the browser source? They should be separate processes.

avrovulcanxh607 commented 2 years ago

OK, I'll report back later - I note there are several obs-browser-page processes running, even though I only have two browser sources, is that normal?

kkartaltepe commented 2 years ago

Please remove all third party plugins from your installation before reporting a memory leak upstream.

kkartaltepe commented 2 years ago

Also provide actual evidence of the OOM killer from your system logs. Or provide an actual crash from obs via gdb.

avrovulcanxh607 commented 2 years ago

Been an hour or two now, Here's the output from Top:

1168540 nathan    20   0 7917808   2.3g 225176 S 156.2   7.3 204:21.59 obs
1168674 nathan    20   0 5336440  85820  70036 S   6.2   0.3   0:37.36 obs-browser-pag
1168594 nathan    20   0  228584  49464  40184 S   0.0   0.2   0:00.00 obs-browser-pag
1168595 nathan    20   0  228584  51056  41736 S   0.0   0.2   0:00.01 obs-browser-pag
1168627 nathan    20   0 1125584 110680  82420 S   0.0   0.3   0:40.36 obs-browser-pag
1168665 nathan    20   0  747800  72332  59944 S   0.0   0.2   0:00.15 obs-browser-pag
1168673 nathan    20   0 5885052 177696  73780 S   0.0   0.5   0:44.47 obs-browser-pag
1168675 nathan    20   0 5316984  44656  33048 S   0.0   0.1   0:00.01 obs-browser-pag
1168738 nathan    20   0 1012176  57296  47504 S   0.0   0.2   1:12.11 obs-browser-pag

And after an hour or so:

1168540 nathan    20   0 9883276   4.2g 225412 S 181.2  13.4 452:39.25 obs
1168594 nathan    20   0  228584  49464  40184 S   0.0   0.2   0:00.00 obs-browser-pag
1168595 nathan    20   0  228584  51056  41736 S   0.0   0.2   0:00.01 obs-browser-pag
1168627 nathan    20   0 1125584 110680  82420 S   0.0   0.3   1:26.19 obs-browser-pag
1168665 nathan    20   0  747800  72332  59944 S   0.0   0.2   0:00.16 obs-browser-pag
1168673 nathan    20   0 5885052 177696  73780 S   0.0   0.5   1:36.12 obs-browser-pag
1168674 nathan    20   0 5336440  85820  70036 S   0.0   0.3   1:22.06 obs-browser-pag
1168675 nathan    20   0 5316984  44656  33048 S   0.0   0.1   0:00.01 obs-browser-pag
1168738 nathan    20   0 1012176  57296  47504 S   0.0   0.2   2:40.27 obs-browser-pag

I don't see any large memory fluctuations in the browser processes, certainly not in the order of 2GB like the main process has...

@kkartaltepe,

Jan 18 21:35:20 NMPServer6 kernel: [425641.019680] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/user@1000.service,task=obs,pid=820881,uid=>
Jan 18 21:35:20 NMPServer6 kernel: [425641.019838] Out of memory: Killed process 820881 (obs) total-vm:26470860kB, anon-rss:20432784kB, file-rss:0kB, shmem-rss:21964kB, UID:1000 pgtables:42956kB oom_score_adj:0
Jan 18 21:35:21 NMPServer6 kernel: [425641.937448] oom_reaper: reaped process 820881 (obs), now anon-rss:16kB, file-rss:0kB, shmem-rss:22732kB
avrovulcanxh607 commented 2 years ago

Just to confirm, the issue still remains with the "third party plugins" absent.

WizardCM commented 2 years ago

I have two more avenues for testing, if you don't mind.

  1. Duplicate the scene collection (using the Scene Collection menu at the top)
  2. After the memory usage is sufficiently high (but not high enough to crash), start deleting sources and see if the memory usage drops (either a little for each source until it normalises, or a large jump with a particular source)

If that doesn't provide an obvious cause, my next suggestion would be to try without any sources. You can switch to a blank scene collection for this test.

avrovulcanxh607 commented 2 years ago

OK, for reference my OBS instance consumes around 2GB starting from cold. This jumps to 6GB when I load my scene collection. After running for a while, the RAM usage reaches 14GB - deleting each scene causes a marginal reduction, as does switching the scene collection. Stopping the live stream causes the largest reduction, however this is still only on the order of 1GB or so, leaving 12GB in use even with a completely empty, or newly created scene collection...

I've also just left it running without actually streaming, that also causes the memory to rise...

I note, however, that starting from cold in a blank scene collection does not cause the memory to rise - as such I suppose it's a matter of starting from cold with a different scene missing each time! I'll get back to you with the results.

WizardCM commented 2 years ago

That is very strange. It does sound like a source may be leaking, and even after deleting/unloading it doesn't clean itself up.

My personal guess at this stage is probably a media source of some kind.

Thank you for the detailed testing & report.

avrovulcanxh607 commented 2 years ago

I suspect either (or both) of two video sources I have that run in a continuous loop. I'll let you know shortly

avrovulcanxh607 commented 2 years ago

OK, I can confirm the issue is with my continuous looping video sources. These are set not to restart when active, and not to close when inactive. This is presumably what causes the issue.

avrovulcanxh607 commented 2 years ago

Well apparently starting a stream causes the issue to return, whether the looping files are present or otherwise... Is there any chance this is somehow related to my system exclusively using CPU for the video rendering etc?

avrovulcanxh607 commented 2 years ago

OK, quick update, after a bit of further testing I can say with a certainty that it's something to do with the looping videos. Interestingly the issue prevails even if I set them to not load when not in use, however this only applies during streaming - AFAICT disabling them while idle doesn't cause any issues. With the two sources removed (and my normal third party plugins re-instated), the system has been running for over 24 hours with no issues - RAM still sitting at 6GB, yay! Would it help if I sent over the offending videos?

Fenrirthviti commented 2 years ago

Definitely could not hurt to have the videos causing issues so we can try and reproduce on our end. It's possible there is an issue with the files themselves, however, but even if there is, I agree that runaway memory usage is something we should be trying to avoid in any case, even if it's not technically "our fault".

ogmkp commented 2 years ago

I had some kind of memory leaks on Debian, back to 2016. But no seeing this anymore.