[BUG] OBS duplicates scenes and corrupts scene collections when using groups

[skeletonbow]

When using groups with nested scenes in OBS, the conditions can arise where OBS will corrupt both the active scene collection as well as any other scene collections that are configured. Over time the corruption gathers multiple duplicate copies of scenes and spreads through the collection until finally OBS starts crashing, and eventually wont even start any more, crashing on start every time. The problem appears to occur primarily with people who use Groups in scenes, which would explain why it isn't more widespread. People who are more advanced users doing more advanced layered/nested scene setups are most likely to encounter this problem.

The problem for a long time was thought to be caused by 3rd party addons but has now been reproduced with pristine OBS installation without any 3rd party plugins/scripts/filters etc. and we have a reproducible minimal test case to provide.

Platform

Operating system and version: Windows 10 x64 (1909) OBS Studio version: 25.0.4

Expected Behavior

OBS should never corrupt the active or inactive scene collection configuration data under any circumstances, when users (advanced or otherwise) are using the program to create scenes using built in features in a manner that it permits.

Current Behavior

OBS will corrupt the scene collection(s) by adding duplicates of a particular scene to the same scene collection, and sometimes spreading those dupes into other scene collections that are not in use, adding scenes that they don't contain, eventually corrupting the entire scene collections which may become unstable, crash and unusable.

https://obsproject.com/logs/msN9RZEizUNnbt7W

Steps to Reproduce

I've been working hard to try to make a reproducible test case for this for a while which did not use 3rd party filters etc. but whenever I thought I had a solid test case, it wouldn't always happen, so I've hesitated to report the bug formally until I could nail a reliable test case that was potentially actionable. I've gotten close recently, but today in Discord user Lindenkron provided a very simple reproducible test case that I was able to reproduce on a fresh pristine OBS install. Here are the steps:

1. Create scene collection "Bug Test 1"
2. Create 2 scenes named "Scene" and "Scene Nested"
3. In "Scene", add a single Group, and then add scene source "Scene Nested" inside the group.
4. Create new scene collection and name it "Bug Test 2", with just the default "Scene" present.
5. Switch back to "Bug Test 1" scene collection.
6. Drag the "Scene Nested" scene to be above "Scene" in the scenes pane.
7. Switch back to scene collection "Bug Test 2"
8. Close OBS, then open OBS back up.

When OBS opens it will be in scene collection "Bug Test 2" which previously only had a single empty scene in it named "Scene". It now has been corrupted to have "Scene Nested" in it also, which was from scene collection "Bug Test 1".

Additional information

If you have multiple groups in use and more complex setup, the amount and level of corruption that occurs becomes much larger, with many scenes being copied into other scene collections, and multiple duplicates of them showing up and spreading throughout different scene collections in OBS, until the program becomes unstable and eventually wont even start anymore, crashing on startup until you delete the Scene collection configuration directory and start over from scratch until the problem starts to happen again.

This problem appears to be related to the use of Groups, which seems to be a feature that has a number of bugs related to it. As a heavy user of this feature I'll try to do my part to file bug reports for any other Group related problems I encounter in the future and hopefully we can work towards stabilizing these awesome features.

Thanks so much to the OBS developers, Xaymar, Surn, Lindenkron, MrGregles, ihazabeard, R1CH and any other creators or developers who have helped to diagnose this problem and/or work on fixing it. This will mean a lot to so many people out there when it's eventually fixed, and will lead to a huge amount of creativity being possible in OBS. Very much appreciated guys, thanks for looking into this!

[skeletonbow]

[notr1ch]

Confirmed here, this seems to be a problem with ClearSceneData not actually clearing all scene data before returning. If other threads are holding references to scenes / sources, or there is a bug with reference counting, then sources won't actually be released and destroyed in ClearSceneData, and the new scene collection's scenes and sources are appended to obs->data.first_source from the current / previous collection. ClearSceneData also doesn't enumerate scenes, only sources, so stale scenes are also present in libobs structures.

Suggested plan: ClearSceneData needs to synchronize and lock audio / video / etc threads to ensure nothing is holding references before the function returns. Call obs_enum_scenes to remove all scenes. I strongly suspect there is a leaking reference somewhere also contributing to this issue, given that the bug reproduces when using groups, the group code seems a likely cause.

[skeletonbow]

Today I've now reproduced this bug without using groups also, just by using nested scenes, and having more than one scene collection present. I'm not sure exactly how or when it happened but it is the same symptoms describe dhere with duplicated scenes with the same names appearing in the scene list, and leaking into other scene collections as well. Not sure if any of the code changes above would reflect on this also or not, but I wanted to add this information to the bug report for now as a data point for completeness. OBS has been reliably crashing on exit every time both before and after the corruption but not sure why that's happening yet either or whether it might be related.

Unfortunately, I do not have a reproducible test case for this, so it's just data for now. I'll provide updated info if I glean anything new though.

[Xaymar]

Just now had this happen by updating OBS Studio to 25.0.8 from 25.0.1. A scene I deleted came back, and now can't be deleted, and persists across scene collections. Edit: This scene was part of a group, but shouldn't be in any groups anymore. It never appeared in 25.0.1.

[ToeFrog]

I experienced this issue this morning after updating to version 25.0.8. I launched OBS and noticed a few of my scenes were duplicated. My setup uses nested scenes as well. I have my main scenes and other scenes labeled as components. Those components are used as scene sources in my main scenes. A few of my scenes were duplicated. After deleting the duplicated scene the one that was left was corrupted and had to be removed and re-created as well.

[therentabrain]

Just adding my data: This is happening increasingly, and it may have to do with duplicate entries for the same scene showing up in scene_order. Looking at the json, it doesn't appear like the scenes are always duplicate, but they are listed multiple times in scene_order.

I think this bug showed up at the same time as another one: The currently loaded scene collection is showing up doubly in the drop-down list. (Windows 7 64)

This is a showstoppingly terrible problem for all the organizations whose webstreams I'm running. All workarounds are so appreciated.

Is there a version of OBS before this started? .1? I'll try downgrading.

Forum thread: https://obsproject.com/forum/threads/scenes-duplicating-when-using-scene-collections.116926/

[notr1ch]

As a workaround, the latest version should no longer persist the deleted items to disk, so switching scene collections and immediately exiting and restarting OBS should be safe.

[therentabrain]

Is that 25.0.8? Or some kind of beta release? I just tried and haven't been able to confirm this. The corrupt scene collection seems to get worse every time I open it. It's hard to be sure of that because there are so many items. It appears to add items to the json file immediately on loading, and again possibly on closing OBS.

This may or may not be related to the file already having corruption. Perhaps it would be kinder to a file that is 100% valid.

It occurs to me that one way to fix without sacrificing the project would be to look at 'name' values and ensure that when there's a conflict, it adds a unique identifier, like -1, -2, etc. Yes I know this will still involve broken references, but I imagine with unique names, at least, I would be able to open each item, fix them, remove them, etc. Right now, there's no way to repair the items, aside from deleting them entirely (when that works). I imagine that's because they have the same name.

All advice welcome; if I succeed in repairing my project I'll be sure to post. Is this the right place to do this? I am relatively new at github. The forum is full of problems and very few solutions. Thanks very kindly.

[therentabrain]

Update: I wrote a super quick processor that looks recursively for duplicate named things at each level and gives them a politely unique name (MySource 1, MySource 2, etc).

Use at your own risk, but so far it seems it saved my butt and I really hope it can save someone else's. I had to recreate a little but the project has been fine so far.

https://therentabrain.com/resources/obsjson/

[notr1ch]

Did some more work on this tonight. Seems like this was introduced in https://github.com/obsproject/obs-studio/commit/c0e2e7f12ebc99ff3c2c64b633c29989692f9912, I expected the QDataStream stuff to be responsible but they work fine with a standard QListWidget.

I'm not sure where to go from here as nothing else about that commit stands out as a potential cause of reference leaks. Perahps @VodBox could take a look at it.

[therentabrain]

FWIW, using .8, after fixing as above, I ran a show with many window-captured guests, scenes, and media files yesterday, using my rescued project. I didn't change scene collections over the weekend; if I needed to switch to a different version of the scenes, I closed OBS and swapped in a different json file. OBS behaved perfectly, aside from three problems.

I'll include them here in case this is at all useful, forgive me/you're welcome if it's unnecessary amounts of info.

One, I copied a scene into another scene, and it created funky extra copies of everything in the scene. I thought that had already been prevented by an update. I probably said Duplicate when I meant Reference. But it was easy to delete the pasted scene and I think the funky copies vanished with it (or I deleted them, I can't remember). Either way, no corruption, no identical names showing up multiply.

Two, this is probably unrelated, but since it has also come up since v25 I thought I might as well mention: Sometimes when adding a source, it adds it but I can't see it in the scene list until I move up and down between scenes. It appears as though the new thing didn't paste, or didn't add, and I am tempted to add it / paste it again, and duplicates could result.

And three, at one point I deleted what I thought was a copy of a source that I believe my util had renamed because it was perceived to be a duplicate - and the entire original source disappeared from all the scenes it was in. This may have been a bad keystroke on my part, it was quite a surprise. I had set up Second Copy to keep archive backups of my json files every 5 minutes, so I was able to go back and it didn't happen again.

If there's a page about rules on what to post/not to post here in github, I haven't found it. :)

[skeletonbow]

As a workaround, the latest version should no longer persist the deleted items to disk, so switching scene collections and immediately exiting and restarting OBS should be safe.

Scene collections appear to have been designed with the idea of being able to switch between them while live without having to end a stream and restart it, which would be quite problematic for many use cases. On Twitch for example, one can expect to lose a large percentage of their viewers potentially even if pre-announcing they're coming right back.

Since scene collections and groups are both very dangerous to use right now due to the data corruption, I would offer an alternative workaround of using multiple parallel installations of OBS with the portable install method instead, and avoiding using scene collections or groups in any of them, sticking to nested scenes and the StreamFX source mirror source which helps to avoid some cases where extra scenes or groups are needed.

This doesn't solve the problem of a live stream being disconnected to switch between setups, but it does greatly reduce triggering groups and scene collection related bugs that cause corruption.

As a separate workaround for the problem of having to disconnect to switch between OBS installs (or restart it), I'd suggest having two copies of OBS running in parallel:

1) One of several parallel installs you switch between instead of using scene collections. This is the main OBS with your normal scenes, plus the NDI plugin set to output OBS master output.

2) A second barebones OBS setup containing a single scene and the NDI plugin, using NDI source to capture the output of the first OBS instance above and stream it to the given service.

This is far less complex than setting up an RTMP server, and the primary OBS can be shut down and restarted with a different configuration or whatever while the 2nd OBS maintains the connection to the service provider, possibly throwing up an ad or 30-60s long transition video clip to keep viewers occupied while the other OBS is restarted and reconnects over NDI.

[skeletonbow]

Two, this is probably unrelated, but since it has also come up since v25 I thought I might as well mention: Sometimes when adding a source, it adds it but I can't see it in the scene list until I move up and down between scenes. It appears as though the new thing didn't paste, or didn't add, and I am tempted to add it / paste it again, and duplicates could result.

I believe I filed a bug report for that also, although I can't seem to find it.

[TreZc0]

This is actual a major bug, and with many workflows, there is not a even a good workaround for it. Can we expect a fix any time soon?

[therentabrain]

FWIW, I've done several complicated shows since my last post. Ever since using my little utility to clean up the duplicate names, and being careful to close and reopen the app after changing scene collections, I haven't had any of this trouble at all.

If anyone else is plagued by these problems, try running your json file through my util and make sure you close/reopen OBS after changing scene collections. It's been good to focus entirely on the art and the message, and not to be arguing with software!

Let me know if I can assist anyone in making my util work for you.

[notr1ch]

Are you able to introduce corruption without moving scenes with the drag / drop?

[therentabrain]

Are you asking me, and if so I'm not sure what you are asking exactly :)

[notr1ch]

I traced the corruption to https://github.com/obsproject/obs-studio/commit/c0e2e7f12ebc99ff3c2c64b633c29989692f9912 which means the scenes / sources should only be corrupted if you actively drag and drop them to change ordering. Using the up / down arrows to move their order should be fine. I'm wondering if there is another path to introduce the corruption other than via the dragging to reorder.

[TreZc0]

FWIW, I've done several complicated shows since my last post. Ever since using my little utility to clean up the duplicate names, and being careful to close and reopen the app after changing scene collections, I haven't had any of this trouble at all.

If anyone else is plagued by these problems, try running your json file through my util and make sure you close/reopen OBS after changing scene collections. It's been good to focus entirely on the art and the message, and not to be arguing with software!

Let me know if I can assist anyone in making my util work for you.

I get that sentiment, but sorry, when working with dynamic graphics through NodeCG for example and using a set of instances for different channels that need to be accessed at the same time, this is a hack, nothing more. Of course, these sources have the same name. They ARE the same, just on different instances. The scenes for example cannot even have different names in our scenario since they have a specific syntax that can be called directly via websocket. That's just not the way it was designed and makes it pretty dangerous to use in any professional 24/7 or close restreaming environment. This bug was introduced fairly recently, and it should have been fixed months ago :/

[therentabrain]

TreZc0: Oh I agree, I was just trying to support anyone who is trying to make shows with the software in its buggy condition. I didn't mean to be correcting you, just offering interim help :)

notr1ch: I haven't had any duplication problems with the current version since I cleaned up my corrupted json. I've had the other problems I've posted about (adding a new source doesn't show up as visible, dragging and dropping changes the vertical scroll position) and a few others I haven't (audio source forgets which device it is set to, window capture rotates itself inexplicably and needs to be reset to 0) but nothing serious. I don't know that dragging and dropping scenes caused my corruption, but I would not be surprised, since I do have a lot of scenes and I try to put them in show-ready order before each show using drag and drop.

[TreZc0]

Any news here? Still quite a breaking bug.

[Xaymar]

If you have StreamFX installed, versions prior to 0.8.2 had a bug that caused scene duplication on complex setups, such as nested scenes. Aside from that, I have only observed this with switching scene collections, but it can be avoided by first switching to an empty scene collection, and then then to the target.

[TreZc0]

That's another workaround however...we're working in a highly complex remote setup where we cannot always see unexpected outcomes the second we switch cause most of it is done via websocket.

To be frank, I don't quite understand how a confirmed bug that can cause permanent damage to your saved scene collections by corrupting source properties is not severe enough to be prioritized...

[Fenrirthviti]

It is a priority, but we can't just wave a magic wand and fix bugs. This is very difficult bug that we have not been able to isolate the cause for. These things take time.

[TreZc0]

Don't get me wrong, I understand the difficulties, especially with an open source project and the amount of bugs coming in everyday. The lack of communication in here just made the issue look forgotten about after 3 months. No hard feelings, I can relate and would love to help if I had the time capacity.

Sorry if it came out wrong.

[skeletonbow]

This isn't intended as a solution for anyone else just to be clear to anyone reading this later, but I have worked around this problem temporarily since I reported it by completely avoiding using the features of OBS that cause the problem. Namely the Groups feature and the Scene Collections feature I no longer use either of them.

Instead of using Scene Collections I now use OBS portable mode, with a single installation and config files/data per scene collection instance. This way each OBS installation has a single scene collection, so no switching occurs and no problems occur triggered by them. The caveat is that I can not switch scene collections while OBS is running, during a streaming or recording session. That is not a problem for me currently, but if I need that functionality in the future I can try setting up a local RTMP server and streaming to that as an intermediary, and then disconnect one OBS instance and reconnect to it with another OBS instance, and theoretically do any recording from the RTMP server side of things instead of within OBS. Not an ideal solution but technically possible workaround if/when I need it.

Groups: I have replaced all usage of groups throughout my OBS scenes with a few different solutions depending on how/why I was using the Groups feature in each instance. Here are my use cases and workarounds for each:

Case 1: Using a Group as a wrapper for a single source in order to apply filters to it in the current scene without applying them to the source directly which would be global. Multiple groups may be used like this to apply different collections of filters or transforms to the same original source/scene. Solution 1: Using the Source Mirror filter from StreamFX to clone the source, and applying the filters to the Source Mirror instead. Where Source Mirror can't be used, I use an OBS Scene as a wrapper and nest it using Scene Source instead. If I need multiple copies of a source, I'll use multiple Source Mirrors or wrapper scenes as appropriate.

Case 2: Using a Group to apply filters/transforms to a number of sources as a group. Solution 2: Using a secondary meta Scene in OBS to collect sources together, and nesting this scene in the main scene with Scene Source, then applying filters to the nested scene.

Case 3: Complex layering of various sources, duplicates each with a number of individual filters applied which all combine to produce an advanced OBS visual effect. Solution 3: Converting such effects to standalone custom pixel shaders wherever possible, replacing all of the OBS source/scene/group/filter complexity of the effect with a single shader filter. This usually improves both performance and configurability also as it is a personally customized solution.

By using these methods since filing the report I've avoided triggering some of the known bugs with these features in OBS, and it has given me a much more stable OBS experience overall by using these workarounds in the mean time. I've helped quite a few other streamers/creators adapt their OBS setup to these methods to avoid some of the known issues and it has worked out well for them as well.

Again though, these methods are not intended nor being suggested as a long term solution to be content with for anyone, and not everyone will necessarily understand or have the appropriate experience/skills to employ these workarounds (in particular writing custom pixel shaders for example), they are potentially useful for those who are able and willing to make use of them in the short term until the issues in question are resolved in some future OBS release.

Having said all that, I'd like to add that I actually decided to learn how to program pixel shaders from scratch due to a combination of the Groups and Scene Collection related bugs, as well as some encouragement by Xaymar for how to take my visual effects etc. to the next level by learning shader programming. It has only been 4 months since I started learning HLSL for just for OBS visual effects and to simplify my configurations and it was the best decision ever. It's not only greatly simplified my overall configuration but has allowed me to create special visual effects otherwise not possible. I am now learning how to create my own LUA scripts to extend OBS, and plan to make native plugins in the future as well.

If it wasn't for the bugs in these features I never would have embarked on these adventures for another year or two likely, but I'm kind of glad now that I did dive in sooner because of them. I'm looking forward to the eventual resolution of the issues and being able to use Groups again some time, but things are going well with the above workarounds for now.

Happy to test any proposed fixes/nightly builds etc. as portable installs if/when they are available. Thanks again for looking into these tricky issues!

[skeletonbow]

Oh, one more thing I'd like to add... In the future another solution for me will be having multi-pass shaders and shaders or filters that can combine the textures of multiple sources all within a single shader source/filter. This will basically allow compositing of scenes within a single shader/filter, offering another solution for grouping things and also providing additional functionality (such as Photoshop style blend modes for example). Not something that can be done with current tools, but probably in the near future.

[AntoineBE]

Scene corruption and duplicated object weird things still happening ... damn frustrating... Happen when duplicated a scene/suppressing an element. Creating a copy of another element.

[skeletonbow]

Scene corruption and duplicated object weird things still happening ... damn frustrating... Happen when duplicated a scene/suppressing an element. Creating a copy of another element.

I think you should probably file a separate bug report and fill out the template with your full details, in order to give the devs the greatest amount of data for trying to diagnose and hopefully fix the issue. It may have a similar result but be a different bug.

[gxalpha]

So far, most of the evidence here suggests that this issue appears to have been caused by an earlier version of a plugin somehow corrupting the scene collection. If the collection is already corrupted when it gets loaded by OBS (even with the plugin uninstalled), there is little we can do.

If it's still possible to corrupt a scene collection (that was not touched by any third-party plugin) in OBS 28, for example by using grid mode, please open a new report with detailed reproduction steps.