search

insidegui / WWDC

The unofficial WWDC app for macOS
https://wwdc.io
BSD 2-Clause "Simplified" License
8.62k stars 780 forks source link

Use time stretching for fast playback rates without audio distortion #486

Closed shadowfacts closed 6 years ago

shadowfacts commented 6 years ago

The WWDC app would benefit from using time stretching when the playback rate is greater than 1 in order to speed up the content without audio distortion or affecting the pitch of the audio significantly.

To compare the differences with/without time stretching, open a video in Chrome and Safari and change the playback rate (the YouTube player has this option, or by modifying the playbackRate JS property of a <video> element from the console) and see how in Chrome the audio still sounds pretty natural, but sounds much more distorted in Safari.

I find audio using time stretching instead of the normal playback rate increase to be much more pleasant to listen to, and I would love for this to be supported in the app.

I believe this is possible using AVAudioUnitTimePitch, but I'm not sure how to use AVFoundation audio units with AVPlayer or integrate it into the app.

insidegui commented 6 years ago

We had someone implement something like this, but we disabled it because of other issues. We're currently working on changes to the player that are probably going to change the way it behaves when playing at faster speeds.

@allenhumphreys Is this something you'd like to take a look at?

allenhumphreys commented 6 years ago

@shadowfacts AVAudioUnitTimePitch appears to be for the audio engineering portion of AVFoundation and there is no apparent API on AVPlayer/Asset/PlayerItem that could make use of it. We are using the time domain-based stretching built into AVFoundation and supported by AVPlayerItem's audioTimePitchAlgorithm property. It is configured on this line

The article cited doesn't clearly explain the why behind the difference in your experiences in Safari and Chrome. It is likely that you're just hearing different implementations of the algorithms mentioned in the article. One thing that is certain is that we are not using the resampling method, as that would result in a very different experience (chipmunks at high speed). You can experience that for yourself by setting the audioTimePitchAlgorithm to .varispeed. (to great comic effect!)

insidegui commented 6 years ago

I'm closing this since it's not an actionable issue.