AVFoundationPlayer: Overall Stability Issues with Recent Developments

[urshofer]

Hello -

I made an application which is asynchronously loading many short movie files (from 1 to 5 seconds a file) over a long period of time. This means countless loadAsync, play and pause commands.

I used a recent git master version https://github.com/openframeworks/openFrameworks/commit/27b09aae8c2108cb78cd1b18140ba6ab1040a891 where @i-n-g-o made some changes and I tested them quite extensively (https://github.com/openframeworks/openFrameworks/pull/3736).

Compiling the same application against current oF master creates certain instabilities and runtime crashes. I'll try to give some more details about the bugs, but I need some time to gather them. I just noticed that since the commit above some major changes were made to the AVFoundationPlayer.

Best, Urs

[i-n-g-o]

a bit more locking and a semaphore helps. pushed: https://github.com/i-n-g-o/openFrameworks/tree/ofAVFoundationPlayer_fastloading_fixes

[ofTheo]

@i-n-g-o just tested your latest changes. I don't get any issues with crashes on close or fast play. nice work!! :)

I tested the memory issue @armadillu mentioned: With the code below I see the memory going up each time I hit 'r' - waiting a while it doesn't seem to be getting released.

Seems really close though! Thanks so much!!

ofAVFoundationPlayer        fingerMovie;
int timeLastLoaded = 0;
//--------------------------------------------------------------
void ofApp::setup(){
    ofBackground(255,255,255);
    ofSetVerticalSync(true);
    frameByframe = false;

    // Uncomment this to show movies with alpha channels
    // fingerMovie.setPixelFormat(OF_PIXELS_RGBA);

    fingerMovie.load("movies/fingers.mov");
    fingerMovie.setLoopState(OF_LOOP_NORMAL);
    fingerMovie.play();
    timeLastLoaded = ofGetElapsedTimef();
}

//--------------------------------------------------------------
void ofApp::update(){
    if( ofGetElapsedTimeMillis() - timeLastLoaded > 500 && ofGetKeyPressed('r')){
        fingerMovie.close();
        fingerMovie = ofAVFoundationPlayer();
        fingerMovie.load("movies/fingers.mov");
        fingerMovie.setLoopState(OF_LOOP_NORMAL);
        fingerMovie.play();
        timeLastLoaded = ofGetElapsedTimeMillis();
    }
    fingerMovie.update();
}

[urshofer]

@i-n-g-o I'm also testing right now your latest branch with my automatic cinema project. this was always a very hard test for the video framework. keep you posted tomorrow.

[i-n-g-o]

thanks for testing! there still some issues with the code. this one for example:

fingerMovie.loadAsync("movies/fingers.mov");
fingerMovie.close();

will load the movie anyways ;) a small thing, but not unimportant. there are a couple of these situations. might have a look at these later the day.

[urshofer]

I guess that in general oF needs a unified approach for async functions. Some trigger events, others not. Strategies like "check if loaded otherwise wait another update/draw cycle" always look pretty tinkered.

[urshofer]

dear all

I was running my application thru the night just to find out that my power supply broke apart. later this morning i encountered some crashes but stupidly i realized that i checked out the wrong oF-branch. stupid me!

the good news are, my application runs smooth for 3 hours now and it looks that there's also no memory leaking with this branch: https://github.com/i-n-g-o/openFrameworks/tree/ofAVFoundationPlayer_fastloading_fixes

I'll keep you noticed if something's happening...

urs

[urshofer]

ok - it's still running smooth. also loading speed seems quick. I would love to see that @i-n-g-o 's branch is merged into 0.9.

[urshofer]

@i-n-g-o I've just encountered a crash. Never had this one before. Probably thread related?

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000028082cc1
Exception Note:        EXC_CORPSE_NOTIFY

External Modification Warnings:
Thread creation by external task.

VM Regions Near 0x28082cc1:
    CG backing stores      0000000012168000-00000000122e8000 [ 1536K] rw-/rw- SM=PRV  
--> 
    MALLOC_TINY            000000007a600000-000000007a800000 [ 2048K] rw-/rwx SM=PRV  

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_pthread.dylib         0x9422c44f pthread_mutex_unlock + 11
1   com.apple.MediaToolbox          0x94e950b6 0x94d03000 + 1646774
2   com.apple.avfoundation          0x956d73fa -[AVAssetReaderOutput copyNextSampleBuffer] + 168
3   com.apple.avfoundation          0x9575d043 -[AVAssetReaderTrackOutput copyNextSampleBuffer] + 73
4   com.urshofer.automaticcinema.player 0x005f9ef5 -[ofAVFoundationVideoPlayer updateFromAssetReader] + 1429 (ofAVFoundationVideoPlayer.m:959)
5   com.urshofer.automaticcinema.player 0x005f9697 -[ofAVFoundationVideoPlayer update] + 119 (ofAVFoundationVideoPlayer.m:793)
6   com.urshofer.automaticcinema.player 0x006912b6 ofAVFoundationPlayer::update() + 54 (ofAVFoundationPlayer.mm:212)
7   com.urshofer.automaticcinema.player 0x005ec628 ofVideoPlayer::update() + 40 (ofVideoPlayer.cpp:201)
8   com.urshofer.automaticcinema.player 0x000f9267 ofxGaplessVideoPlayer::update() + 1559 (ofxGaplessVideoplayer.cpp:223)
9   com.urshofer.automaticcinema.player 0x0010113d ofApp::update() + 4477 (ofApp.cpp:488)
10  com.urshofer.automaticcinema.player 0x0062044a std::__1::__function::__func<std::__1::shared_ptr<of::priv::Function<ofEventArgs, std::__1::recursive_mutex> > ofEvent<ofEventArgs, std::__1::recursive_mutex>::make_function<ofBaseApp>(ofBaseApp*, void (ofBaseApp::*)(ofEventArgs&), int)::'lambda'(void const*, ofEventArgs&), std::__1::allocator<std::__1::shared_ptr<of::priv::Function<ofEventArgs, std::__1::recursive_mutex> > ofEvent<ofEventArgs, std::__1::recursive_mutex>::make_function<ofBaseApp>(ofBaseApp*, void (ofBaseApp::*)(ofEventArgs&), int)::'lambda'(void const*, ofEventArgs&)>, bool (void const*, ofEventArgs&)>::operator()(void const*&&, ofEventArgs&) + 42 (ofEvent.h:237)
11  com.urshofer.automaticcinema.player 0x005f0f48 ofEvent<ofEventArgs, std::__1::recursive_mutex>::notify(void const*, ofEventArgs&) + 248 (functional:1756)
12  com.urshofer.automaticcinema.player 0x005eef7c ofCoreEvents::notifyUpdate() + 44 (ofEvents.cpp:200)
13  com.urshofer.automaticcinema.player 0x0067d858 ofAppGLFWWindow::update() + 24 (ofAppGLFWWindow.cpp:348)
14  com.urshofer.automaticcinema.player 0x0061d6b1 ofMainLoop::loopOnce() + 177 (memory:3919)
15  com.urshofer.automaticcinema.player 0x0061d5d8 ofMainLoop::loop() + 40 (ofMainLoop.cpp:122)
16  com.urshofer.automaticcinema.player 0x005eb2d6 ofRunApp(std::__1::shared_ptr<ofBaseApp>) + 86 (ofAppRunner.cpp:161)
17  com.urshofer.automaticcinema.player 0x005eb22a ofRunApp(ofBaseApp*) + 74 (ofAppRunner.cpp:141)
18  com.urshofer.automaticcinema.player 0x00130bc2 main + 82 (main.cpp:17)
19  libdyld.dylib                   0x97a066ad start + 1

[armadillu]

oh something I forgot about is this new feature XCode7 has, "address sanitizer". It might help find sneaky bugs... Are you using it in your tests? It's been reaaaally helpful for me.

[i-n-g-o]

@urshofer in case of the crash: can you say if this happened around the time a movie was loaded?

[urshofer]

@i-n-g-o that's very hard to say. but I'm pretty sure it was while loading...

[i-n-g-o]

just pushed some changes: https://github.com/i-n-g-o/openFrameworks/tree/ofAVFoundationPlayer_fastloading_fixes

changes: assign operator fix video = ofAVFoundationPlayer(); does not leak anymore change how videoplayer destruction works cleanups

@urshofer can you start a test again?

[urshofer]

hi @i-n-g-o

after a comparison between the (in the meantime) quite old version https://github.com/openframeworks/openFrameworks/pull/3736 and the one here I see that memory consumption differs. I guess it's leaking somwhere although instruments don't show me where. The old one was running stable using around 150mb, whereas this version constantly adds a little bit. Since I've missed the latest developments I'm not sure where the additional memory usage is coming from.

for oF 0.9, I suggest to include the version https://github.com/openframeworks/openFrameworks/pull/3736 in the release. this one is stable. all later commits could be merged into master afterwards.

[ofTheo]

@i-n-g-o @urshofer - I don't see any memory leaks with the current branch. I did a bunch of tests with both the new player and super fast close/load as well as async.

Everything seemed really good to me! I would suggest PR-ing this into OF for 0.9.0

Could you put together a PR? Thanks so much for busting this out!

[i-n-g-o]

compared to #3736 there are quite some improvements this this branch. it might be that the version of the branch is using more memory because of additional async dispaches but there should not be any leaks.

3736 does have some serious problems, which i hope to have improved in the branch.

i am just about to finish things up and will create a PR asap.

[i-n-g-o]

PR is out: https://github.com/openframeworks/openFrameworks/pull/4535

[bilderbuchi]

closed by #4535

[i-n-g-o]

related issue: https://github.com/openframeworks/openFrameworks/issues/4538

[urshofer]

running tests with the oF master merge I'm still encountering troubles I did not have before.

• memory consumption is constantly growing (probably due to related issue https://github.com/openframeworks/openFrameworks/issues/4414)

• crashes after a while (compiled on 64bit) with the following message

2015-11-11 14:55:15.935 AC Player[36201:4130144] AVPlayerItem cannot service a seek request with a completion handler until its status is AVPlayerItemStatusReadyToPlay.
2015-11-11 14:55:15.936 AC Player[36201:4130144] (
    0   CoreFoundation                      0x00007fff89261e32 __exceptionPreprocess + 178
    1   libobjc.A.dylib                     0x00007fff86f63dd4 objc_exception_throw + 48
    2   AVFoundation                        0x00007fff97922633 -[AVPlayerItem seekToTime:toleranceBefore:toleranceAfter:completionHandler:] + 1860
    3   AVFoundation                        0x00007fff9798de0a -[AVPlayer seekToTime:toleranceBefore:toleranceAfter:completionHandler:] + 140
    4   AC Player                           0x00000001004dd72a -[ofAVFoundationVideoPlayer seekToTime:withTolerance:] + 730
    5   AC Player                           0x00000001004dd364 -[ofAVFoundationVideoPlayer seekToStart] + 68
    6   AC Player                           0x00000001004dd26d -[ofAVFoundationVideoPlayer togglePlayPause] + 77
    7   AC Player                           0x000000010001b808 _ZN21ofxGaplessVideoPlayer6updateEv + 808
    8   AC Player                           0x0000000100023184 _ZN5ofApp6updateEv + 3908
    9   AC Player                           0x0000000100500b25 _ZNSt3__110__function6__funcIZN7ofEventI11ofEventArgsNS_15recursive_mutexEE13make_functionI9ofBaseAppEENS_10shared_ptrIN2of4priv8FunctionIS3_S4_EEEEPT_MSE_FvRS3_EiEUlPKvSG_E_NS_9allocatorISL_EEFbSK_SG_EEclEOSK_SG_ + 37
    10  AC Player                           0x00000001004d41c1 _ZN7ofEventI11ofEventArgsNSt3__115recursive_mutexEE6notifyEPKvRS0_ + 257
    11  AC Player                           0x00000001004d1fb6 _ZN12ofCoreEvents12notifyUpdateEv + 22
    12  AC Player                           0x000000010055f217 _ZN15ofAppGLFWWindow6updateEv + 23
    13  AC Player                           0x00000001004fd22a _ZN10ofMainLoop8loopOnceEv + 186
    14  AC Player                           0x00000001004fd148 _ZN10ofMainLoop4loopEv + 40
    15  AC Player                           0x00000001004ce2c6 _Z8ofRunAppNSt3__110shared_ptrI9ofBaseAppEE + 86
    16  AC Player                           0x00000001004ce222 _Z8ofRunAppP9ofBaseApp + 66
    17  AC Player                           0x0000000100052302 main + 66
    18  libdyld.dylib                       0x00007fff862555ad start + 1
    19  ???                                 0x0000000000000003 0x0 + 3
)
2015-11-11 14:55:16.035 AC Player[36201:4130144] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'AVPlayerItem cannot service a seek request with a completion handler until its status is AVPlayerItemStatusReadyToPlay.'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff89261e32 __exceptionPreprocess + 178
    1   libobjc.A.dylib                     0x00007fff86f63dd4 objc_exception_throw + 48
    2   AVFoundation                        0x00007fff97922633 -[AVPlayerItem seekToTime:toleranceBefore:toleranceAfter:completionHandler:] + 1860
    3   AVFoundation                        0x00007fff9798de0a -[AVPlayer seekToTime:toleranceBefore:toleranceAfter:completionHandler:] + 140
    4   AC Player                           0x00000001004dd72a -[ofAVFoundationVideoPlayer seekToTime:withTolerance:] + 730
    5   AC Player                           0x00000001004dd364 -[ofAVFoundationVideoPlayer seekToStart] + 68
    6   AC Player                           0x00000001004dd26d -[ofAVFoundationVideoPlayer togglePlayPause] + 77
    7   AC Player                           0x000000010001b808 _ZN21ofxGaplessVideoPlayer6updateEv + 808
    8   AC Player                           0x0000000100023184 _ZN5ofApp6updateEv + 3908
    9   AC Player                           0x0000000100500b25 _ZNSt3__110__function6__funcIZN7ofEventI11ofEventArgsNS_15recursive_mutexEE13make_functionI9ofBaseAppEENS_10shared_ptrIN2of4priv8FunctionIS3_S4_EEEEPT_MSE_FvRS3_EiEUlPKvSG_E_NS_9allocatorISL_EEFbSK_SG_EEclEOSK_SG_ + 37
    10  AC Player                           0x00000001004d41c1 _ZN7ofEventI11ofEventArgsNSt3__115recursive_mutexEE6notifyEPKvRS0_ + 257
    11  AC Player                           0x00000001004d1fb6 _ZN12ofCoreEvents12notifyUpdateEv + 22
    12  AC Player                           0x000000010055f217 _ZN15ofAppGLFWWindow6updateEv + 23
    13  AC Player                           0x00000001004fd22a _ZN10ofMainLoop8loopOnceEv + 186
    14  AC Player                           0x00000001004fd148 _ZN10ofMainLoop4loopEv + 40
    15  AC Player                           0x00000001004ce2c6 _Z8ofRunAppNSt3__110shared_ptrI9ofBaseAppEE + 86
    16  AC Player                           0x00000001004ce222 _Z8ofRunAppP9ofBaseApp + 66
    17  AC Player                           0x0000000100052302 main + 66
    18  libdyld.dylib                       0x00007fff862555ad start + 1
    19  ???                                 0x0000000000000003 0x0 + 3
)
libc++abi.dylib: terminating with uncaught exception of type NSException
(lldb)