search

CrendKing / avisynth_filter

DirectShow filters that put AviSynth and VapourSynth into video playing
MIT License
107 stars 8 forks source link

Random hanging #41

Closed PhantomGamers closed 3 years ago

PhantomGamers commented 3 years ago

Environment

Describe the bug

After watching a video for a period of time (it doesn't seem very consistent, sometimes it happens within 5 minutes, sometimes within 15, ~15 minutes tends to be the longest), with madVR the video frame freezes but the audio continues to play, and pausing the video and replaying it causes it to start playing video again but no longer at the interpolated framerate (not positive what is happening here, is it SVP's script? avisynth filter? avisynth+? How can I find out?) When using EVR the entire player just crashes. I have zero problems using SVP with just ffdshow (well, as far as crashes/hanging goes).

I'd love to figure out how to get this working as the one major change over ffdshow that helps so much for the experience is that with ffdshow, the video will freeze for a bit after AR changes and SVP's black bar detection kicks in. With avisynth filter it transitions nearly seamlessly.

I thought maybe this could have been related to #24 but I've tried messing with the CPU and GPU queue sizes and it seems to happen regardless of what they are set to.

I should also mention that I tried it with the AvySynth+ version that gets installed by SVP (3.5 I think?), and it also occurred there, I manually updated it to 3.7.0 seeing that it had improvements for multithreading and thought maybe it would help, but alas it did not.

Checklist

avsf.zip (log file)

CrendKing commented 3 years ago

Without a consistent repro, I can't pinpoint the root cause. But I found a pattern in your log file that shares with https://github.com/CrendKing/avisynth_filter/issues/40. I already fixed that problem in https://github.com/CrendKing/avisynth_filter/commit/fe73211d792b8b4d827f0c35d31f77f4bee7b29b. Can you try AviSynthFilter.zip?

PhantomGamers commented 3 years ago

Without a consistent repro, I can't pinpoint the root cause.

Yeah I know, it's really annoying on my end as well, but it's never consistent. It'll hang, and if I restart the video file from the beginning it'll hang at a completely different part...

And it still happens with the version you've posted here.

Something else I've noticed, using ffdshow instead of avisynth filter I sometimes get frames with green artifacts (it happens so fast that I'm unable to get a screenshot of it), so I'm wondering if perhaps there's a problem somewhere else in the pipeline that ffdshow just gracefully recovers from whereas avisynth filter does not?

Here are some more logs of freezes in case it helps anything avsf.zip

CrendKing commented 3 years ago

I found a concurrency issue for #40. I'm not sure if it's the same issue, but it doesn't hurt trying: AviSynthFilter.zip.

PhantomGamers commented 3 years ago

I think I can confirm that this is some issue between avisynth filter and SVP's black bar detection & removal.

I watched through a whole show and it didn't freeze once, and what I've noticed is that there were zero A/R changes throughout, whereas the other 2 shows I was watching had quite a few.

I didn't consider trying this before because as I said, I wasn't getting crashing with ffdshow, but after making the link I went back to the most crash-prone parts of the other shows and didn't get any crashing with the black bar detection disabled.

I've attached another avsf log file as well as the event log from SVP (no errors in there). Not sure if I should also post this on the SVP forums / through SVP or if continuing this discussion here is appropriate? @chainikdn (avsf log) avsf.zip (svp log) active.zip

CrendKing commented 3 years ago

Interesting. Can you post screenshot of your SVP settings (especially the black bar stuff) so I can replicate the environment?

PhantomGamers commented 3 years ago

image image

Everything should be default except for the frame conversion being set to "To screen" (I have a 100Hz monitor)

Perhaps this is being a little overcautious, but in case some DMCA bot picks up the names, imdb:tt9140560 and imdb;tt1898069 were the key offenders in triggering this (for the latter, it seemed especially bad in 4k but I also had some issues with 1080p sources so that might be irrelevant I'm not positive)

Also for the latter, S03E05 has a ton of A/R changes so it's really easy for me to reproduce with that

CrendKing commented 3 years ago

I never used the black bar feature. I see that some part of S03E05 has no black bars and then later there are. SVP seems to automatically detects these bars on realtime and adjust. Does it hang when some of these change happen?

PhantomGamers commented 3 years ago

It automatically detects the changes in realtime and adjusts, but it usually doesn't happen immediately on a switch, usually it will switch and actually continue playing for a few seconds (sometimes minutes, even, which is another reason I had trouble tying the two together). The part from about 36:30 to 41:00 is the worst of it, seemingly guaranteed to hang (though not always at the same exact part).

CrendKing commented 3 years ago

I played 4 times with different setup and no hang at all. But more importantly, why would anyone want to use that black bar feature during this scene? Isn't it extremely annoying to see the video size constantly changing and stutter?

chainikdn commented 3 years ago

why would anyone want to use that black bar feature during this scene? Isn't it extremely annoying to see the video size constantly changing and stutter?

off-topic here, but this feature is either on or off... I don't know how it could ignore this scene only though it's set to "one time only" by default meaning it will cut black bars one time at the beginning of playback

CrendKing commented 3 years ago

though it's set to "one time only" by default meaning it will cut black bars one time at the beginning of playback

That makes sense. I can't find any toggle to force it one time only. Is it in one of the advanced "Application settings"?

chainikdn commented 3 years ago

there's a popup menu on the "Black bars detection" button ;)

CrendKing commented 3 years ago

Oh. What a sneaky menu! It is default to Tracking mode though: Clipboard01

@PhantomGamers So changing to "One time only" mode may be a workaround for you now until we gather more info on repro?

chainikdn commented 3 years ago

It is default to Tracking mode though

mmm... could be :D I'm still not sure which one should be the default and it used to change several times in the past

the main problem with black bars is they lead to nasty gray artifacts on the edges

but from the AVSF's point of view it's just a regular output format change, nothing more

PhantomGamers commented 3 years ago

But more importantly, why would anyone want to use that black bar feature during this scene? Isn't it extremely annoying to see the video size constantly changing and stutter?

Yes, during this scene of course I wouldn't want the black bar detection to be on, but it was hanging in basically every episode of the show even where it would only transition once on a scene change. I just mentioned this scene specifically because it seems to be the most consistent at hanging for me, though I'm unsure why it is not hanging for you then hmm

EDIT: When you tried to repro, did you use a 4k source? It could potentially be related if you didn't

CrendKing commented 3 years ago

When you tried to repro, did you use a 4k source? It could potentially be related if you didn't

No. I could try it.

chainikdn commented 3 years ago

I can't find s3 in 4k o_O

PhantomGamers commented 3 years ago

I can't find s3 in 4k o_O

I couldn't find every episode in 4k but S03E05 was one of the ones that I could.

If either of you have discord I can send it to you directly, my discord is the same as my Github #9208

CrendKing commented 3 years ago

Does it have 1,633,697,175 bytes? I played several times with those tons of AR changes and had no hang at all. I did untick P010 and P016 in AVSF settings to force LAV to convert it to 8-bit YV12 format.

PhantomGamers commented 3 years ago

Does it have 1,633,697,175 bytes?

The 4k sources I'm using are around the 12-13GB range, with that specific episode being 14,590,133,052 bytes

How can you change the AVSF settings while using it with SVP? When I try to untick those settings and save it says "AviSynth script file is currently managed by remote control. Your change If any is saved but not used."

I don't think this show is available in HDR though, maybe it is but I don't have an HDR monitor and the freezing occurs even with HDR related settings disabled in SVP.

If neither of you can reproduce this issue, it could be a hardware issue on my end. I've just upgraded my cpu+mobo (the issue was occurring prior, but I didn't report it until after I upgraded because my old cpu was faulty and I thought it might have been that). And I might have gotten a RTX 3080 for close to MSRP (we'll see if it ships lol), so if/when I get that I'll be able to rule out the GPU as being problematic. Then the only thing remaining would be my RAM, which I have tested extensively and I do not believe it to be problematic.

Also I just clean installed Windows when I got the new cpu/mobo, out of an abundance of caution, so if it's not a hardware issue and neither of you can reproduce it, I'm not sure what kind of software configuration would cause such a thing... hmm

CrendKing commented 3 years ago

How can you change the AVSF settings while using it with SVP

The "Input Formats" below is orthogonal to the avs script and can be set independently.

Disaer commented 3 years ago

Hello. I also got frequent crashes while watching 4K HDR with the "Decrease to screen" SVP option enabled, especially on seek or audio track change. Is this bug already reproduced or additional samples are needed?

PhantomGamers commented 3 years ago

I also got frequent crashes while watching 4K HDR with the "Decrease to screen" SVP option enabled

Yes! It seems this alone is responsible for it. I can't seem to reproduce the crashing/hanging after switching it to "Keep normal size."

EDIT: Nevermind, I spoke too soon. With it set to "Keep normal size" I can't reproduce the hang in the aforementioned section of S03E05 but I'm watching a 1080p source of S03E10 currently and it froze again... so that was either only one part of it, or it's just a coincidence that I can no longer seem to reproduce the hang in that segment... very strange.

CrendKing commented 3 years ago

Try AviSynthFilter.zip. May not do anything for you guys. Let me know the result.

PhantomGamers commented 3 years ago

I can still trigger the freezes in "Decrease to screen size" mode with that version, I don't have a reliable repro without that setting.

I think the hangs that happen in "Decrease to screen size" mode and the hangs that happen sporadically are separate issues as I was getting hanging last night while watching the latest episode of imdb:tt6470478 which doesn't have any AR changes.

I should have saved the avsf log of that but I didn't, I will try and get one when I have a chance.

CrendKing commented 3 years ago

Is every video that hanged 4K? Could it be that too much memory were used so that additional memory can't be allocated?

Also, MPC-HC has a feature that when some error happens, it displays a message box with some info. It also has crash reporter (but I have no access to it. clsid should have). So maybe you can try it and somehow get some info?

Also, if you are developer and have Visual Studio, when it hangs, you can attach VS and pause to get stack trace.

PhantomGamers commented 3 years ago

Nope, the sporadic crashes happen with 1080p videos as well! It just seems to happen more often at 4k.

When I have a chance I will try and get a stack trace of the hang.

CrendKing commented 3 years ago

Does v0.9.5 Beta fix the problem?

PhantomGamers commented 3 years ago

Sadly not.

I've made a dump file of the mpc-be process after a hang, not sure if this will help you. https://pixeldra.in/u/wH9L2srk

and here's another avsf log just in case

EDIT: Also I just realized I never responded to your MPC-HC suggestion. Is that feature you mentioned enabled by default? Or where is it in the settings?

If it's enabled by default then the hang is not triggering an error within the player because I get the same exact issue within MPC-HC and never saw any errors output from it.

CrendKing commented 3 years ago

I've made a dump file of the mpc-be process after a hang, not sure if this will help you. https://pixeldra.in/u/wH9L2srk

If you told me you are capable of doing crash dump yourself, we could have made progress long time ago.

If it's enabled by default then the hang is not triggering an error within the player because I get the same exact issue within MPC-HC and never saw any errors output from it.

If there's no error that's fine. I was asking because the hang could be caused by an exception in concurrency, which should trigger crash handler to generate dump.

But if you can capture dump right at hang, then I can give you a debug build. I'll keep its pdb for analysis.

Hope you have a good fiber network for another 600MB upload ;-)

Try debug version first. But if somehow it does not trigger hang anymore, try release version.

avisynth_filter.zip

PhantomGamers commented 3 years ago

If you told me you are capable of doing crash dump yourself, we could have made progress long time ago.

This was my first time having to manually create a crash dump, as opposed to one being generated automatically, so I wasn't sure that I was capable of it myself until you suggested attaching the vs debugger lol

Hope you have a good fiber network for another 600MB upload ;-)

I have a gigabit connection but sadly I cannot upload to pixeldrain at those speeds! If I had a way of communicating with you directly I could send it to you directly through my hfs server but I would not want to publish my IP address here.

Here is the dump file with the debug build (it's twice as large because I did it with the reproducible segment which is from a 4k source, the one I posted yesterday isn't so reproducible (it'd take like 30 minutes to hang).

And here's another avsf.log

Also I suppose this probably isn't directly related to the hanging, since like I said it also happens in sources that have no a/r changes, but this is what gets thrown when a/r changes happen:

Exception thrown at 0x00007FFF8A1CD759 in mpc-be64.exe: Microsoft C++ exception: IScriptEnvironment::NotFound at memory location 0x000000230757C3E0. Exception thrown at 0x00007FFF8A1CD759 in mpc-be64.exe: Microsoft C++ exception: IScriptEnvironment::NotFound at memory location 0x000000230757C3E0. Exception thrown at 0x00007FFF8A1CD759 in mpc-be64.exe: Microsoft C++ exception: ReturnExprException at memory location 0x000000230757D3C0. Exception thrown at 0x00007FFF8A1CD759 in mpc-be64.exe: Microsoft C++ exception: [rethrow] at memory location 0x0000000000000000. Exception thrown at 0x00007FFF8A1CD759 in mpc-be64.exe: Microsoft C++ exception: ReturnExprException at memory location 0x000000230757D3C0.

CrendKing commented 3 years ago

Try avsf.zip

PhantomGamers commented 3 years ago

With that version, it hangs immediately with any source.

Exception thrown at 0x00007FFF8A1CD759 in mpc-be64.exe: Microsoft C++ exception: ReturnExprException at memory location 0x00000070B944D3D0. Exception thrown at 0x00007FFF8A1CD759 in mpc-be64.exe: Microsoft C++ exception: [rethrow] at memory location 0x0000000000000000. Exception thrown at 0x00007FFF8A1CD759 in mpc-be64.exe: Microsoft C++ exception: ReturnExprException at memory location 0x00000070B944D3D0.

avsf.log

And the dump file

CrendKing commented 3 years ago

My bad. I'll give you another one.

BTW, those exceptions are bogus. They are seemingly harmless. So forget about them.

CrendKing commented 3 years ago

Lots of changes in this one. I thought one line was enough. Turns out it causes dead lock thus hang. I need to revamp the whole synchronization mechanisms in avsf.

Expect crash, but mainly let me know if it still hangs.

AviSynthFilter.zip

PhantomGamers commented 3 years ago

Yes, it still hangs with that version. Additionally, I managed to get my hands on a 3090 so I can confirm it's not the old GPU that was responsible for the hanging. The only thing that remains in my system is the RAM, which I've tested extensively so I don't think it's related to that.

Dump file avsf.log

CrendKing commented 3 years ago

The hang happened at exactly the same location, while now a lock is held in the new version. So I have no idea anymore why it hangs when we deliver frames to madVR for rendering.

I could try a different lock.

Does it hang if you use EVR? If yes, please upload a dump.

PhantomGamers commented 3 years ago

It does hang in EVR, which is progress from 0.94 which would just straight up cause the player to crash in EVR!

Dump file avsf.log

CrendKing commented 3 years ago

The EVR hang is caused by AviSynth+, which I have no control of.

And I'm confused with your madVR avsf.log. It doesn't seem in sync with that uploaded dump.

PhantomGamers commented 3 years ago

The EVR hang is caused by AviSynth+, which I have no control of.

I can open an issue over there then. Is there anything specific you noticed that I should pass along?

And I'm confused with your madVR avsf.log. It doesn't seem in sync with that uploaded dump.

My bad, not sure what happened. Here is another dump/avsf.log with madVR.

Dump avsf.log

CrendKing commented 3 years ago

Try avisynth_filter_64.zip

PhantomGamers commented 3 years ago

Still hangs Dump avsf.log

CrendKing commented 3 years ago

What about this? Also if madVR hangs, please try EVR. avisynth_filter_64.zip

PhantomGamers commented 3 years ago

Both madvr and evr still hang.

I figured out why the avsf log and dump file didn't match up before, it's because sometimes when mpc crashes the process doesn't actually terminate and so I was attached to the wrong process. Or at least I think that is what happened. These should hopefully be good.

evr dump evr avsf.log

madvr dump madvr avsf.log

CrendKing commented 3 years ago

I'm sorry the dumps don't seem to match. Could you please upload again with this file? avisynth_filter_64.zip

But from the log I don't think my fix actually matters if the hang happens there. Or it could be that there were multiple sources of hangs, and we fixed some while others still exist (the earlier hangs would have "hidden" later hangs), because I no longer see a hang location in your recent uploads that I once saw in your earlier upload.

CrendKing commented 3 years ago

Another version avisynth_filter_64.zip

CrendKing commented 3 years ago

FYI, I fixed issue 24, which could help eliminating some hang. You might want to try.

PhantomGamers commented 3 years ago

I will test soon! I've been busy with midterms so haven't had a chance to test your changes.

ssssssbbb commented 3 years ago

Lots of changes in this one. I thought one line was enough. Turns out it causes dead lock thus hang. I need to revamp the whole synchronization mechanisms in avsf.

Expect crash, but mainly let me know if it still hangs.

AviSynthFilter.zip

With this version, I can prevent hanging by setting the "rendering device" to "CPU only" in SVP, no matter how much the queue size is.

Hanging comes much faster in every version after this. The last version in issue 24 is even worse, it hangs every time I right click in PotPlayer.

Please tell me how to produce dump or log. But 我传不了mega，能QQ或度盘吗？

Disaer commented 3 years ago

I've sent a sample via PM on the Doom9 forum. It crashes when setting the "Decrease to screen" SVP option.