search

shaka-project / shaka-player-embedded

Shaka Player in a C++ Framework
Apache License 2.0
239 stars 62 forks source link

Playback freezes after seek #93

Closed jgongo closed 4 years ago

jgongo commented 4 years ago

I'm doing preliminary tests of the player to integrate it in our application. During those tests I tried to find out how to seek to a given point and tried to update the currentTime property. I found that after updating it the player would seek to the requested point, but after reaching the point where the seek was requested, the video would freeze. To be clear, with the code below:

I'm using the framework with the default configuration (configure --ios) running in the simulator, with one of the Shaka demo assets (the Star Trek one).

This is the code I used to test the seek functionality:

class ViewController: UIViewController, ShakaPlayerClient {
    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)

        // Make a Shaka Player view.
        let player = ShakaPlayerView()
        player.setClient(self)
        player.frame = self.view.bounds
        self.view.addSubview(player)

        // Load and play an asset.
        player.load("https://storage.googleapis.com/shaka-demo-assets/angel-one/dash.mpd", withStartTime: 0) { error in
            if let error = error {
                NSLog("Error loading resource: \(String(describing: error.message))")
            } else {
                NSLog("Resource correctly loaded")
            }
            player.play()
        }
        DispatchQueue.main.asyncAfter(deadline: .now() + 15) {
            player.currentTime = 0
        }
    }

    func onPlayerError(_ error: ShakaPlayerError!) {
        NSLog("Error while playing: %@", error.message)
    }

    func onPlayerBufferingChange(_ is_buffering: Bool) {
        NSLog("Buffering changed: \(is_buffering)")
    }

    func onPlayerPlayingEvent() {
        NSLog("Player is playing")
    }

    func onPlayerPauseEvent() {
        NSLog("Player is paused")
    }

    func onPlayerEndedEvent() {
        NSLog("Player ended")
    }

    func onPlayerSeekingEvent() {
        NSLog("Player seeking started")
    }

    func onPlayerSeekedEvent() {
        NSLog("Player seeking ended")
    }
}

And here is the resulting log:

[Info]: "Starting attach..."
[Info]: "Starting load of https://storage.googleapis.com/shaka-demo-assets/angel-one/dash.mpd..."
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.20.08.00.02.02.02.00 bandwidth=333746 frameRate=25 width=320 height=240"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=1100994 frameRate=25 width=768 height=576"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.10.08.00.02.02.02.00 bandwidth=114126 frameRate=25 width=192 height=144"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.21.08.00.02.02.02.00 bandwidth=612642 frameRate=25 width=480 height=360"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=829676 frameRate=25 width=640 height=480"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=97374 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=152000 channelsCount=2"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.20.08.00.02.02.02.00 bandwidth=333746 frameRate=25 width=320 height=240"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=1100994 frameRate=25 width=768 height=576"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.10.08.00.02.02.02.00 bandwidth=114126 frameRate=25 width=192 height=144"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.21.08.00.02.02.02.00 bandwidth=612642 frameRate=25 width=480 height=360"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=829676 frameRate=25 width=640 height=480"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.20.08.00.02.02.02.00 bandwidth=333746 frameRate=25 width=320 height=240"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=1100994 frameRate=25 width=768 height=576"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.10.08.00.02.02.02.00 bandwidth=114126 frameRate=25 width=192 height=144"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.21.08.00.02.02.02.00 bandwidth=612642 frameRate=25 width=480 height=360"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=829676 frameRate=25 width=640 height=480"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.20.08.00.02.02.02.00 bandwidth=333746 frameRate=25 width=320 height=240"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=1100994 frameRate=25 width=768 height=576"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.10.08.00.02.02.02.00 bandwidth=114126 frameRate=25 width=192 height=144"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.21.08.00.02.02.02.00 bandwidth=612642 frameRate=25 width=480 height=360"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=829676 frameRate=25 width=640 height=480"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.20.08.00.02.02.02.00 bandwidth=333746 frameRate=25 width=320 height=240"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=1100994 frameRate=25 width=768 height=576"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.10.08.00.02.02.02.00 bandwidth=114126 frameRate=25 width=192 height=144"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.21.08.00.02.02.02.00 bandwidth=612642 frameRate=25 width=480 height=360"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=829676 frameRate=25 width=640 height=480"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=148953 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=434667 channelsCount=6"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.20.08.00.02.02.02.00 bandwidth=333746 frameRate=25 width=320 height=240"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=1100994 frameRate=25 width=768 height=576"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.10.08.00.02.02.02.00 bandwidth=114126 frameRate=25 width=192 height=144"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.21.08.00.02.02.02.00 bandwidth=612642 frameRate=25 width=480 height=360"
[Log]: "Dropping variant - video not compatible with platform"  "type=video codecs=vp09.00.30.08.00.02.02.02.00 bandwidth=829676 frameRate=25 width=640 height=480"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Dropping variant - audio not compatible with platform"  "type=audio codecs=opus bandwidth=151620 channelsCount=2"
[Log]: "Found variant with audio and video content, so filtering out audio-only content in all periods."
2020-01-23 12:28:09.770956+0100 ShakaPlayer[66219:991576] Buffering changed: true
[Log]: "codecs" "avc1-mp4a" "avg bandwidth" 3978617.2
[Log]: "onChooseStreams_"   {startTime:0, textStreams:[...], variants:[...]}
[Log]: "Choosing new streams after period changed"
[Log]: "init: completed initial Stream setup"
2020-01-23 12:28:10.028983+0100 ShakaPlayer[66219:991576] Resource correctly loaded
[Log]: "(audio:14)" "looking up segment:"   "presentationTime=0"    "currentPeriod.startTime=0"
[Log]: "(video:7)"  "looking up segment:"   "presentationTime=0"    "currentPeriod.startTime=0"
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd4b083e000] Protocol name not provided, cannot determine if input is local or a network protocol, buffers and access patterns cannot be configured optimally without knowing the protocol
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0123 12:28:10.243077 148815872 ffmpeg_decoder.cc:250] No hardware-accelerators available, using decoder: aac_at
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd4b0828200] Protocol name not provided, cannot determine if input is local or a network protocol, buffers and access patterns cannot be configured optimally without knowing the protocol
[Log]: "(video:7)"  "startup complete"
[Log]: "(all) setting up Period 0"
[Log]: "(all) Stream 7 is being or has been set up"
[Log]: "(all) Stream 14 is being or has been set up"
[Log]: "(all) Period 0 is being or has been set up"
I0123 12:28:10.314819 150425600 ffmpeg_decoder.cc:256] Using decoder: h264, with hardware accelerator: videotoolbox
[h264 @ 0x7fd4b3007600] Failed setup for format videotoolbox_vld: hwaccel initialisation returned error.
E0123 12:28:10.466975 150425600 ffmpeg_decoder.cc:222] Hardware pixel format is unsupported, may be falling back to software decoder.
2020-01-23 12:28:10.484394+0100 ShakaPlayer[66219:991576] Player is playing
2020-01-23 12:28:10.504849+0100 ShakaPlayer[66219:992125] [plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x600002778b20> F8BB1C28-BAE8-11D6-9C31-00039315CD46
2020-01-23 12:28:10.533577+0100 ShakaPlayer[66219:991576] Buffering changed: false
[Log]: "canSwitch_"
[Log]: "Calling switch_(), bandwidth=23989 kbps"
[Log]: "switch_"
[Log]: "switch: switching to Stream (video:9)"
[Log]: "(video:9)"  "looking up segment:"   "presentationTime=4"    "currentPeriod.startTime=0"
[Info]: "Aborting current segment request to switch."
[Log]: "switch: Stream (audio:14) already active"
[Log]: "(video:9)"  "looking up segment:"   "presentationTime=4"    "currentPeriod.startTime=0"
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd4b0828200] Found duplicated MOOV Atom. Stopping.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd4b0828200] Protocol name not provided, cannot determine if input is local or a network protocol, buffers and access patterns cannot be configured optimally without knowing the protocol
I0123 12:28:13.229717 150425600 ffmpeg_decoder.cc:256] Using decoder: h264, with hardware accelerator: videotoolbox
[h264 @ 0x7fd4af8eac00] Failed setup for format videotoolbox_vld: hwaccel initialisation returned error.
E0123 12:28:13.232420 150425600 ffmpeg_decoder.cc:222] Hardware pixel format is unsupported, may be falling back to software decoder.
[Log]: "Calling switch_(), bandwidth=16752 kbps"
[Log]: "switch_"
[Log]: "switch: Stream (video:9) already active"
[Log]: "switch: Stream (audio:14) already active"
2020-01-23 12:28:24.475052+0100 ShakaPlayer[66219:991576] Player seeking started
[Log]: "(all): seeked: buffered seek: presentationTime=0.001"
2020-01-23 12:28:24.476664+0100 ShakaPlayer[66219:991576] Player seeking ended
2020-01-23 12:28:24.476802+0100 ShakaPlayer[66219:991576] Player is playing
I0123 12:28:38.444981 150425600 ffmpeg_decoder.cc:256] Using decoder: h264, with hardware accelerator: videotoolbox
[h264 @ 0x7fd4b0022c00] Failed setup for format videotoolbox_vld: hwaccel initialisation returned error.
E0123 12:28:38.447176 150425600 ffmpeg_decoder.cc:222] Hardware pixel format is unsupported, may be falling back to software decoder.
[Log]: "Calling switch_(), bandwidth=12459 kbps"
[Log]: "switch_"
[Log]: "switch: Stream (video:9) already active"
[Log]: "switch: Stream (audio:14) already active"
[Log]: "Stall detected at 14.933999999999871 for 1.001999855041504 seconds. Seeking forward 0.1 seconds."
[Log]: "(all): seeked: buffered seek: presentationTime=15.033999999999871"
2020-01-23 12:28:40.566134+0100 ShakaPlayer[66219:991576] Player seeking started
I0123 12:28:40.571604 150425600 ffmpeg_decoder.cc:256] Using decoder: h264, with hardware accelerator: videotoolbox
[h264 @ 0x7fd4b0022c00] Failed setup for format videotoolbox_vld: hwaccel initialisation returned error.
E0123 12:28:40.572661 150425600 ffmpeg_decoder.cc:222] Hardware pixel format is unsupported, may be falling back to software decoder.
[Log]: "Stall detected at 15.033999999999871 for 1 seconds. Seeking forward 0.1 seconds."
2020-01-23 12:28:41.813986+0100 ShakaPlayer[66219:991576] Player seeking started
[Log]: "(all): seeked: buffered seek: presentationTime=15.13399999999987"
I0123 12:28:41.845192 150425600 ffmpeg_decoder.cc:256] Using decoder: h264, with hardware accelerator: videotoolbox
[h264 @ 0x7fd4b39bdc00] Failed setup for format videotoolbox_vld: hwaccel initialisation returned error.
E0123 12:28:41.846360 150425600 ffmpeg_decoder.cc:222] Hardware pixel format is unsupported, may be falling back to software decoder.

The last lines (stall detected/seeking forward) seem to be repeated until presentationTime reaches the end of the video

jgongo commented 4 years ago

FYI, I have tried seeking forward and it does not work (it immediately freezes). Is this the correct way to seek? I see no other way to do it with the exposed interface, am I missing anything?

jgongo commented 4 years ago

I've successfully run the sample app, and this problem doesn't seem to happen in it, although the code seems to be doing the same as I do, this is, setting the currentTime property of the ShakaPlayerView. Any idea why this is happening?

TheModMaker commented 4 years ago

Are you trying with the latest from master? I'm in the process of rewriting the media pipeline and have made major changes in the last month or so. I cannot reproduce this error with your code. It seems to work fine for me.

You can also pass GLOG_v=1 as an environment variable to get more verbose logging from C++; or you can set player.logLevel = .V1 to get more verbose logging from the JavaScript player.

jgongo commented 4 years ago

I was using the version from January 7th. After updating to the latest from master the video no longer freezes, but the audio consistently stops playing using the Angel One asset (#97).