search

mixxxdj / mixxx

Mixxx is Free DJ software that gives you everything you need to perform live mixes.
http://mixxx.org
Other
4.53k stars 1.28k forks source link

Some SoundSourceProxyTests fail on Win11 with VS2022 #11094

Open JoergAtGithub opened 1 year ago

JoergAtGithub commented 1 year ago

Bug Description

I build the same versions of Main (9965d55) and PR #11074 on two Windows laptops. On the older one with Win7 and VS2019 all tests passed. But on the newer one with Win11 and VS2022 four tests failed:

SoundSourceProxyTest.seekForwardBackward 
SoundSourceProxyTest.seekBoundaries
SoundSourceProxyTest.readBeyondEnd
SoundSourceProxyTest.regressionTestCachingReaderChunkJumpForward

The test SoundSourceProxyTest.seekForwardBackward generated ~2GByte of similar log messages, here is shortened version of the log file: LastTest_soundsourceproxytestsfailed_newbuilenv.log

Version

Main

OS

Windows11 + VS2022

uklotzde commented 1 year ago

SoundSourceMediaFoundation is hardly working due to lack of Windows developers/testers and might fail when Microsoft decides to either change the API or behavior.

The API itself is not really suitable for sample-accurate decoding.

uklotzde commented 1 year ago

https://www.reddit.com/r/DJs/comments/z9f7yf/comment/iygulke/

https://community.native-instruments.com/discussion/7760/3-7-0-336-rc-uploaded

Known Issues that we’re still working on:

  • Audio glitches on M4A files introduced by the latest Windows 11 update.
daschuer commented 1 year ago

Does the issue move with the binary, or is it an issue of the windows version running on?

Since this issue makes the cue points and waveforms unreliable we may consider to retire MediaFoundation and move to FFMPEG for aac/mp4/m4a decoding.

Unfortunately there is a high probability that the cues and waveforms will be shifted for all such files after the switch.

To be aware of that I have prepared: https://github.com/mixxxdj/mixxx/pull/4773 waiting for merge.

A next step would an automatic adjustment of these offsets.

JoergAtGithub commented 1 year ago

I just updated to the latest Windows11 update - these tests still fail.

daschuer commented 1 year ago

Without rebuilding?

JoergAtGithub commented 1 year ago

I now copied all .exe and .dll files from the build directory of my Windows7 laptop to the build directory of my Windows11 laptopand ran CTest. On Windows7 all tests with these executables were pass, on Windows11 the 4 SoundSourceProxyTests failed.

Conclusion: It's an unfixed OS bug in Windows11 and not a Mixxx bug.

daschuer commented 1 year ago

Great, thank you. for investigations. Does anyone know how to report a bug upstream? Even if there is a chance that it is fixed some day, we will have users suffering this issue.

JoergAtGithub commented 1 year ago

These tests still fail, using latest Win11.

daschuer commented 1 year ago

@JoergAtGithub: Does Windows 11 find the Sound Start Cue created in Windows < 11? Just play one or more m4a file you had in your Windows before the OS Update.

and look for First sound has been moved!

Than please Build Mixxx with ffmpeg https://github.com/mixxxdj/mixxx/blob/a547664420463515926a619aafe08b378b34e035/src/sources/soundsourceffmpeg.cpp#L470 = Highest

And verify it again.

daschuer commented 1 year ago

The Test failure is reproducible on the GitHub Workflows using Windows-2022: https://github.com/daschuer/mixxx/actions/runs/5855464923/job/15873332041

JoergAtGithub commented 1 year ago

The Test failure is reproducible on the GitHub Workflows using Windows-2022

That means that also Windows Server 2022 is affected: https://github.com/actions/runner-images/blob/main/images/win/Windows2022-Readme.md

daschuer commented 1 year ago

As expected the windows-2022 build with FFmpeg succeeds https://github.com/daschuer/mixxx/actions/runs/5865154501/job/15901524981

The crucial question is if the MediaFoundation Windows 10 decoded files match FFmpeg decoded files perectly, or if we need to consider a cue point shift.

@JoergAtGithub: did you found time to check for the "First sound has been moved!" log entry with m4a files? Interesting would bee if you see that in the FFmpeg build linked above.

I think we should establish a unit test for it.

daschuer commented 1 year ago

https://github.com/mixxxdj/mixxx/pull/11887 Discovered that moving to ffmpeg is not a trivial solution, because there is a timing offset between the FFmpeg and Media Foindation that varies across tracks.

@JoergAtGithub I am now clueless how to continue.

Since I am not on windows I have hard time to investigate that issue further.

It looks like Media Foundation 10.0.20348.1 introduces an offset after seeking. The question is if this offset is constant so that we can compensate that within Media Foundation.

Can you test that with a track with a perfect beat grid before the update and test it on Win11? Has the track always the same offset after the first seek?

Is the issue notable?

Is it an option to release Mixxx 2.4 with this issue included? Shall we move to FFmpeg and force a reanalysis for m4a tracks?

Ideas?

daschuer commented 1 year ago

To continue here we need a Windows user with a Windows 11 machine that has a significant amount (> 10) m4a tracks analysed with WIN10 or before. Who can help here?

JoergAtGithub commented 1 year ago

I could to the analysis task, since I've both, a Win7 and Win11 laptop with latest Mixxx builds. But I don't have any m4a tracks. Does anybody know, where I could download suitable test files?

daschuer commented 1 year ago

Thank you. You can just convert a few. I think Audacity can do it. It uses Ffmpeg.

daschuer commented 11 months ago

It still fails with Windows 2022: https://github.com/daschuer/mixxx/actions/runs/7285956895/job/19853789720

daschuer commented 11 months ago

The Preview Windows 10 is also failing :-(

10.0.26016.1000 fail 10.0.22621.2506 fail 10.0.22000.1880 fail 10.0.20348.1fail 10.0.20348.2031fail
10.0.20348.2655 fail 10.0.19041.3636 fail 10.0.19041.746 fail

10.0.17763.2989 OK

daschuer commented 11 months ago

Tested with: https://github.com/mixxxdj/mixxx/issues/12289#issuecomment-1867834433

daschuer commented 7 months ago

Do we have a working Windows version in the meantime?

JoergAtGithub commented 7 months ago

No, this test is not fixed in any Windows 11 version known to me.

EchoCoder1729 commented 2 months ago

How are we planning to fox this? Any leads so that I can take this up and maybe fix this test? I am ready to dive into the Windows 11 Sound Driver API calls if necessary. But if this test is going to be breaking across windows versions, then doesn't make much sense to work on it though. Just to clarify, each platform (Linux/ Windows) uses their own test suites or do they run the same tests?

JoergAtGithub commented 2 months ago

The soundsource code tested here is platform specific. In general we have only one test suite.

daschuer commented 2 months ago

I have just pushed a new CI build to check the current situation with Windows 2022 https://github.com/daschuer/mixxx/actions/runs/10733009450

It is Windows Server 2022, Version 21H2: 10.0.20348 Build 2655 https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md

daschuer commented 2 months ago

No :-( https://github.com/daschuer/mixxx/actions/runs/10733009450/job/29765629719

EchoCoder1729 commented 2 months ago

Do we have any environment variable to specify that it is a windows machine and disable the specific test in case windows? Is it that we need to modify the test or any changes would break the test in previous versions like windows 7?

daschuer commented 2 months ago

Windows is really broken here. So the failure of the test is correct. You may disable/skip the test if you like.