Sampler (Dunne): Assertion failure (segments) in `DunneCoreMultiSegmentEnvelopeGenerator::skipEmptySegments()`

[maksutovic]

Describe the bug When attempting to play a note with the Sampler() object, the app crashes. I.E. sampler.play(noteNumber: velocity: channel)

It is a failed assertion on segments in the envelope generator:

Assertion failed: (segments), function skipEmptySegments, file EnvelopeGeneratorBase.cpp, line 93.
dyld4 config: DYLD_LIBRARY_PATH=/usr/lib/system/introspection DYLD_INSERT_LIBRARIES=/Developer/usr/lib/libBacktraceRecording.dylib:/Developer/usr/lib/libMainThreadChecker.dylib:/Developer/Library/PrivateFrameworks/DTDDISupport.framework/libViewDebuggerSupport.dylib:/usr/lib/libMTLCapture.dylib

This is with samples loaded into the sampler via the SFZ method (the preferred method). I have tried loading a sample with Sampler(sampleDescriptor: file) however I am unable to produce any output with this method.

To Reproduce I have made a rudimentary repo which demonstrates the issue: https://github.com/maksutovic/DunneSamplerCrash-AK-5.2.2

1. Clone Repo
2. Run in either Simulator or Device
3. Assertion failure

Expected behavior I expect the Sampler() to in fact play the note specified and not crash.

Screenshots If applicable, add screenshots to help explain your problem.

Details (please complete the following information):

• Type: iPhone 11 Pro OR Simulator
• OS: iOS 15.1 19B74
• AudioKit Version: 5.2.2

Additional context I would be happy to try and resolve this issue however I would definitely need some pointers of how the envelope generator works with the sampler.

[maksutovic]

I have been trying to decode this crash. Where I am now is somehow segments property is not getting set during the init process, thus the assertion fails. If we remove the assertion, we get out of bounds error in the skipEmptySegments() function.

However a majority if not all of the segments are init'ed correctly, there just seems to be be the last segment that .reset() is not called, thus the pointer is null.

If we just return false upon a NULL segment in this method, the sampler never sounds.

If anyone could please give me some pointers, pun intended how I can more accurately trace down this erroneous empty segment I would be so thankful.

[maksutovic]

I believe I have found the culprit.

It seems to me that akSamplerLoadCompressedFile() is not in fact loading any samples. To be more clear, the samples are "loading" and all of the relevant data structures are assigning memory to their data, however any thing relating to the raw samples contained in .wv files are all 0. It seems to me it is not in fact anything to do with the envelope generator, rather segments is NULL because all of the samples are 0 thus there is no attack segment to play.

I have checked with all versions of Sampler() since 5.0.0 -- all of them have the same crash.

The biggest difference for me is that I had Sampler() working since januarary and up to October -- the difference now is being on Xcode 13 (I'm still on Big Sur) -- something with Xcode 13 broke Wavpack properly unpacking samples from .wv files and I believe this is the culprit.

I was able to get the current version of Sampler() working with the documentation provided here: https://audiokit.io/Packages/DunneAudioKit/ under the heading SampleDataDescriptor and loadRawSampleData().

So the mystery remains is why wave pack is not loading samples properly from Xcode 12, to Xcode 13?

[getdunne]

This is a very interesting finding. The Wavpack code is very old-fashioned C/POSIX code, so it would not surprise me if a compiler update resulted in a problem of this kind. I'm still not set up to debug it myself at this time.

[eljeff]

Following along closely. If you can write a test that clearly demonstrates something failing, that's a huge step towards getting it solved. Cause then it's all self contained and anybody can run it w/ the click of a button and see it fail. And more importantly see it succeed, once it's fixed.

We were screen sharing once a few months ago, and we hit this crash early on. Someone uttered those famous last words, 'thats an easy fix', but we decided to table it... it wasn't happening every time (actually it seemed random + rare).

Saying that out loud, it makes me wonder if it is indeed wv related (unlikely I was testing with wavpack), or instead more closely tied to the env generator in general? Can't wait to see what it is.

[maksutovic]

I would be happy to write a test. Could you point me in the right direction regarding how to place this package in the appropriate container? If I see this correctly, this branch just holds the package, but no real means of running it (inside a workspace etc.). Would I branch from here and the add the appropriate tests? Thanks so much.

[eljeff]

I'd jump into the repo for DunneAK, take a look at the other tests there for inspiration. There is already an audio file in the TestResources* folder there, so maybe place a single wv file there for testing, along with your needed sfz definition (honestly just getting the resources to load properly sounds like it will be the most confusing part). Might make sense to create a similar test that loads the already-included wav file, and works. That way there is a working test to compare to.

One other thing, your current project is pretty good but I can't really isolate if it's working at all or not. I added some code that bypasses the CRASH, but I'm not hearing any sound. So it might help too to have the demo have two buttons, one that shows the code working (makes sound, maybe using a wav file?), and another that crashes.

*pro-tip... in a swift package, if you put your resource files in a folder called 'Resources', you can get very confusing errors. Hence why we named the folder TestResources. Tripped me up quite a bit when I made my first package.

[maksutovic]

Oh this is so helpful Jeff, thank you a million! I'm wrapping up a few things and in a few hours I'll get a test up for us!

[eljeff]

I know I'm getting into the weeds here w/ more and more requests, but I just been down this road before so got a few ideas what can help make things go smoother.

When making the test, I think if we keep it as close to the other tests as possible, that helps eliminate variables and simplify things. So maybe use the audio file there (12345.wav), make a wv version of that, and then have your 'sfz' reference that. Basically, re-create the resources to use what we already have, instead of pulling in new stuff we can't vouch for. All I'm saying is, the more variables we eliminate, the better.

ps. I tried your demo project and tried playing a trumpet note just to make sound, and I couldn't hear anything (in the simulator). So I'm already hitting that confusion I mentioned... I don't have a working baseline to compare to.

[maksutovic]

No problem at all, I think that makes perfect sense, I just saw the testing folder, I know exactly how to do it and keep things predictable.

Regarding the trumpet that was a ooof on my part, will have this sorted with the official testing shortly.

[eljeff]

The plot thickens... I am currently hitting this bug via the simulator in a very specific case... sending MIDI. This is on a new machine too, so my simulator might be running a different iOS than I have been, because I have used this setup a week ago with no issues. If I play the note via an on-screen keyboard I have, it doesn't crash. If I send a note via an external controller MIDI, I get the assert in skipEmptySegments.

[maksutovic]

Oh that is so strange, what makes less sense is how would the external midi have anything to do with the sampler? I wrote the tests for SFZ however running the test does not exhibit the crash so I don't know what the utility of the tests are. I have been swamped with other work but I will make some amendments to my sample repo and provide a few buttons with different sampler methods/loading -- I will mimic the tests as exactly as I can to keep continuity.

[aure]

Are you using AudioKit develop branch? We fixed a midi bug a couple of days ago that I need to release a version of AudioKit for.

[maksutovic]

This specific Dunne crash was happening on main branch, honestly I haven't tried the develop branch! I was wondering if that weird midi bug was related in some bizarre way. Let me know how I can be helpful to your release Aure.

[eljeff]

Yea I gotta say the tests are less helpful if they don't exhibit the behavior... that's kind of the point :]. If you can get that other project updated to have a baseline of WORKING and NOT WORKING, that'd be cool. Not too crazy on the UI ... as long as it crashes, and as long as you can also hear it actually working, that'd be cool. I'd love to test my 'fix' on that, but can't until the 'working' part of the project works.

Also w/ the MIDI error I encountered (consistently) earlier, I'm starting to think this isn't going to have anything to do w/ file formats, but is instead a thread thing.

[maksutovic]

I'm sorry for the delay on this, but I finally simplified the dummy project which can be found here: https://github.com/maksutovic/SamplerSanityCheck

There are two buttons: one that plays back a Sampler() object loaded with raw samples (using the 12345.wav from the Dunne TestResources. The other from a SFZ.

The raw sampler does in fact play with no issues, its only the SFZ loaded sampler that crashes.

[eljeff]

The project does indeed build and it's helpful because I can actually hear it work now, thank you. It also crashes just like you describe, so that's awesome.

One question - why are you loading the SFZ on the main thread? I can remove the main thread stuff and it still behaves the same.

[eljeff]

I made some changes via the GitHub web interface... (didn't know you could do that or how well that works). Feel free to edit or reject.

I was able to get 50 lines of your code down to 1. I also removed the Main thread stuff to keep it simpler.

Thank you again - you are clearly demonstrating a major problem. Here's hoping someone who knows what's up takes a look.

[maksutovic]

There was some very stupid reason why I was loading them on the main thread which I cannot remember now -- nothing to do with the library, sorry about that!

Your change to loading the rawSamples is brilliant thank you!

Regarding the crash, I'm still unsure why segments would be nil from the sfz. Was there some undocumented iOS15 stuff that would change the sampler core? I'm not sure if anyone could chime in and see if iOS 14 is working still -- my guess is that it still works for iOS 14/Xcode 12.

[eljeff]

For historical perspective, remember that I've been dealing with this bug for many months now (before ios15)... not in the same way you get it (using sfzs), but in other ways.

We hit it one night when we were fixing something else. I have been hitting it recently by sending MIDI (turns out not as consistently as I originally thought).

It's definitely not exactly the same thing, but it does crash at exactly the same place (assert(segments) in skipEmptySegments()), so that tells me there's just something going on with that part of the code.

[eljeff]

How did that go w me editing your repo from the web? Did it make a PR? Are you able to merge those changes in? I've just never done it that way so very curious...

[aure]

Closing, please create a new issue if the problem still exists.