Shuts down encoder PC

[lindenkron]

Doesn't leave a crash, just kills the application on the encoder PC after 30-60 sec of running.

[fzwoch]

Any more infos? What versions, what OS? Any steps how to reproduce?

[lindenkron]

27.1.3 (Both pcs) Windows 10 (Both pcs)

Fresh OBS (Outside of NDI plugin) on Broadcast Machine.

Just installed plugin (simple), added the feed (showed up instantly) looked too good to be true.

Ran it for a while (since I have NDI/SDI massive sync issues with OBS), was going to do a 2 hour test to see if the audio desync'd or not.

3 times in a row it shut down after about a minute.

Tried 90,95 & 100 quality.

[fzwoch]

And I assume 0.2.0 of this plugin? I may take a look this weekend. I wonder if you could find out if its a crash or one of the panics in my code. I'm not a good Windows guy maybe there is a console log to read somewhere? I just hope I can reproduce..

[lindenkron]

Yes, downloaded about 30 mins ago to see if it could save my Broadcast for tomorrow.

There's nothing I can see on my end, other than it shutting off. There's no logs or crash logs.

I inquired about the possibilities of me somehow seeing what's happening, but I was told it's not possible due to what I assume is on the plugins end (calling exit()).

Jim (the creator of OBS) said the following. Hopefully it's helpful to you:

they're not performing crashes correctly it sounds like. or a dependency they're using is calling exit() you're supposed to use the bcrash function you can't really do anything something calls exit() if even exits when a debugger is active it just hard shuts down the program forcibly my suspicion would be that they're using a dependency and the dependency is throwing a crash

[fzwoch]

Any more infos, are you using filter or output mode? Color space and resolution settings?

[lindenkron]

Basic output mode. Both OBS are NV12, 709 color space. Partial range.

1080p60, Bilinear.

[fzwoch]

Can you check if:

1) The same happens when running the output at 1280x720? 2) Check what happens when running the attached version?

I did add some bcrash() functions to locations where OBS may report something when it crashes at these points. However I'm a bit skeptic, since none of these locations seem to be relevant once the stream is running, but you never know..

So far I could not reproduce the issue with any of my Windows 10 machines or VMs.

Maybe you can attach an OBS log to have a look at the specs if anything else may catch my eye.

test.zip

[fzwoch]

P.S. What is "Encoder PC" in your case here? Where the Teleport output is running, or where you receive that stream?

[lindenkron]

Apologies. "Encoder PC" is the PC with the OBS receiving the Teleport feed.

I tested again, 1 minute in with the new version the OBS shut itself off once again. This time on a clean scene collection, I've attached the log. This was still 1080p.

Other things that seem to not be working on PC:

1. Tools > Teleport 'identifier' does not show up under 'Add source > Teleport' as an identifier.
2. When adding a Teleport source, you have to press refresh first to see any feeds (it doesn't seem to fetch any sources on initially being added).

2022-01-23 14-41-20.txt

I'll try 720p when I get time to change settings, but I need my setup in about 60 mins. Some time after tonight I'll upload a log with it.

[fzwoch]

• Tools > Teleport 'identifier' does not show up under 'Add source > Teleport' as an identifier.
• When adding a Teleport source, you have to press refresh first to see any feeds (it doesn't seem to fetch any sources on initially being added).

That is expected behavior. Finding streams takes some time. That is why the refresh button is there for. Alternatively I would have to block the GUI for a couple of seconds. I don't want this to happen. Similar when you change the Identifier name. You need to refresh the list after a couple of seconds to see the effect propagating (also some left over may still be seen while updating the name. This is just as it is)

When the source is the problem, here is another test binary with some changes that tries to recover more faithfully in some situations. The old test may still be worthwhile to check if it reports something.

test2.zip

[fzwoch]

1. Tools > Teleport 'identifier' does not show up under 'Add source > Teleport' as an identifier.

That was actually true. It required a manual output restart to take effect. Will be changed in a future version.

[lindenkron]

1. Tools > Teleport 'identifier' does not show up under 'Add source > Teleport' as an identifier.

That was actually true. It required a manual output restart to take effect. Will be changed in a future version.

Glad to hear.

I just tested 720 on sending PC (Settings > Video > Output (Scaled) Resolution > 720p) 3 minutes; no shutdown.

I disabled teleport (tools > teleport), changed outscale to 1080p - and it instantly killed my OBS (on sending PC).

I opened OBS on sending PC again, restarted OBS on receiving PC and let it run for 5 minutes. Nothing happened, no shutdowns.

I set it to 1080p, restarted both OBS, and started the broadcast again. Ran it for 4 minutes. Nothing happened, no shutdowns.

Not sure what condition is causing these things, with no errors or logs it's really hard to pin point.

Hope this helps in any way, shape or form.

[fzwoch]

Hope this helps in any way, shape or form.

Yes it does. Thanks for testing it out.

I disabled teleport (tools > teleport), changed outscale to 1080p - and it instantly killed my OBS (on sending PC).

I have discovered the same. And hopefully fixed that already. When you change the output settings without restarting OBS the output module crashes (as it was not aware of the change).

I will make a new release in the next few days where I tried to recover from some obscure stream corruptions and addressed all the other things you have noticed so far (I hope). Would be great if you could give it a spin again then.

I'm really worried about the crash while streaming. Because in theory that should never happen as when it runs its really simple. It may be that some network traffic or memory gets corrupted and that concerns be a bit (may still be just a simple bug though).

Or some bug in the SSE/AVX routines for your specific CPU..

If you ever experience the crash after some time it is running and nothing gets changed let me know.

[fzwoch]

https://github.com/fzwoch/obs-teleport/releases/tag/0.2.1 has been released.

[lindenkron]

Have been unable to break it running for an hour now. Unfortunately, it has gone from about 20ms, to 2000ms delay between what's happening on the Sending PC compared to the Receiving PC.

The audio however from my initial testing appear to still keep audio sync - which both SDI & NDI have failed to accomplish. I'll try and keep this thread updated after I've done more extensive testing.

Will have our other broadcaster run an overnight test on an alternative windows broadcast setup to see if it stays alive over night, and have them record a video of the receiving PC to check the audio syncing.

[lindenkron]

6 hours in - no shutdown.

However, unfortunately, the audio has desync'd from the video feed.

I'm seeing a lot of issues with NDI, SDI and various other things and people complaining desync.

But I also find it hard to believe all 3 different solutions, SDI, NDI & Teleport, are flawed. And the only common denominator here is either my machine/network or the OBS application. With other people having these issues, I'm inclined to believe it's the latter.

I enabled the timestamp; which lead to a massive 2+ second delay. I then disabled it again; and suddenly the audio was in perfect sync again.

I really wonder where this drift is coming from, and why no solution seem to be able to keep the sync.

Will keep updated if I come across more issues with shutdown etc. Hopefully I'll have a word from the other broadcaster tomorrow.

[fzwoch]

At least the shutdowns seem gone which from my point of view is a great relieve.

As for the sync - Do you use the output or filter? Sync can only be maintained I believe when audio and video are coming through one source, else it may be difficult to OBS to maintain.

And then again, how do you measure audio sync? Do you check in a final recording, or do you set the source to monitoring? I think monitoring is "broken" - meaning the actual sync is done internally after monitoring outputs the source audio, so I would not rely on it for sync. the truth is in the recording/stream imo.

[lindenkron]

Quick note: I miswrote 'enabled' instead of 'disabled' in this sentence - in case it has any bearing:

I enabled the timestamp; which lead to a massive 2+ second delay. I then disabled it again; and suddenly the audio was in perfect sync again.

I'm using the output, not filters.

I always do a local recording on the receiving PC; then play that back. I use footage that I know have sync on sending PC with clear speech (a youtube clip of people talking).

I check the clip initially, and the sync is good.

I wait a few hours, sometimes more sometimes less. I do the same thing again. Desync on receiving PC.

I enabled, as mention, the timestamp option and then disabled it again - and instantly the sync was back to perfect. Which in itself is sort of strange, if it's OBS (sending PC) that's messing with time stamps. But I don't know the technical side of things.

[fzwoch]

When you toggle the timestamp options it basically reconnects the stream. So it is as you tune in from new.

Increasing latency and sync are two different issues btw. I have no idea if OBS has something in place for clock drift between these two PCs. I assume it doesn't, so latency increasing over time is definitely possible.

[lindenkron]

When you toggle the timestamp options it basically reconnects the stream. So it is as you tune in from new.

Increasing latency and sync are two different issues btw. I have no idea if OBS has something in place for clock drift between these two PCs. I assume it doesn't, so latency increasing over time is definitely possible.

It both increased latency (by about 1 second, unrelated to desync) and desync'd the audio from the video.

Both were perfect (50~ ms and no desync) at the start.

SDI mostly keeps the latency to 50ms~ but also desyncs (in my estimate it was about 400-500ms) over time. NDI has a whole worm of issues variating from desync to delay.

It would make sense to me if the clock drifting is an OBS thing; and all the solutions rely on there being something like that to keep things sync'd but it doesn't exist. Then they all drift eventually?

[lindenkron]

I know we've steered off the topic at hand a bit. If you wish for me to open a new issue for it I will.

Can you explain why this happen; and possible what could be done to account for it/remedy it? https://streamable.com/7cbb3l

This was recorded after approximately 9 hours (7 hour broadcast~)

[fzwoch]

Yeah, if you think the shutdown issue is gone feel free to close.

In the above video - I guess the question is why is the latency different when restarting the stream, right? (Small note, the timestamp option will go away with the next update, but selecting disabled and switching back to the stream should result in the same behavior). This is something I have noted before in other plugins of mine too. I must say I have no idea if it is something on the plugin side. My guess is that it has something to do with OBS calculating the initial latency. But I have no idea what impacts this calculation. Eventually it makes a difference whether audio or video gets send first, or how long the it takes before a video and audio packet has been send. I could not find anything in the documentation that gave me a hint about that. If there is a condition that needs to be taken care of I could try to control it, but I would need to know about it first, which I currently don't.

[lindenkron]

If there is a condition that needs to be taken care of I could try to control it, but I would need to know about it first, which I currently don't.

Have you tried asking in the Dev channel of the OBS Discord? I could ask in there but I'm hardly an expert nor a programmer and I'm not sure their replies would either mean anything to me, or I'd have the knowledge to know what further inquiries to have to any reply I may get.

[fzwoch]

For sure, the main issue normally is that i work on that on my spare time after work. Concentration and the will to debug deep into these things is low ;-)

I have also seen when you switch streams is that the audio latency increases. Looks like a jump in timestamps when streaming is not really handled, or not handled as we like it to be. So best bet atm is to set everything up, restart OBS and let it run without tinkering with the stream selection.

I guess one would have to rewrite timestamps so that they appear continuous to OBS when switching streams or switching to "disabled". I could not spot an internal "reset" API that would force OBS to resync the current source..

[fzwoch]

Closing this one in favor of other tickets.

[fzwoch]

P.S. The original issue may have fixed by d8ce39a7201fd53a5bc9fd65008e9c55bfc322ed

EDIT: Or maybe not.. the original logic looked odd.. but seemed correct nonetheless.