Pitch adjustment

[ryanheise]

Is your feature request related to a problem? Please describe.

Pitch adjustment would be useful in some apps (e.g. changing pitch of voice or music).

Describe the solution you'd like

player.setPitch(1.0); // 1.0 being normal pitch

Describe alternatives you've considered

There is no other way to achieve this other than plugin support.

Additional context

This is easily implemented on Android, iOS might be more problematic (post below if you are able to share useful resources/documentation)

[nt4f04uNd]

i think you should add it to android and then some time later implement it on iOS

[ryanheise]

True.

I should probably mention that all new features from this point will only be developed on the null-safe branch (i.e. the 0.7.* releases). For now, this will require the beta channel of the Flutter SDK, although stable is coming soon.

[ryanheise]

Note to self. This API might integrate well with AVQueuePlayer:

https://developer.apple.com/documentation/avfoundation/avaudiomixinputparameters/1387042-audiotimepitchalgorithm?language=objc

(Though I'm not particularly happy with the quality of iOS's pitch/time stretching algorithms, if the above works, I may as well use it.)

[leidig54]

This would be another massive winner for me. Any eta? As with the equalizer functionality, happy to offer a bounty to bump the priority.

[ryanheise]

Hi @leidig54

Thanks again for offering to support the project!

I will ordinarily prioritise features based on the number of thumbs up and their difficulty, although I also tend to prioritise requests from GitHub sponsors too (otherwise, you're welcome to discuss by email - the address can be found in the git log).

The next two big features I plan to work on are the visualizer and equalizer, but I may throw the pitch shifting feature into that group, too, since there may be some overlap in the section of code involved on the iOS side.

[ryanheise]

It definitely has overlap with what's happening on the visualizer branch and I want to ensure I get the overall design right in how these features interact, so I'm looking into this now.

[ryanheise]

As I originally suspected, this feature would best be handled by a move to AVAudioEngine on iOS.

Although there might be ways to get this to work with AVQueuePlayer, maybe it is finally time to make the switch to AVAudioEngine, given that a lot of the popular feature requests at the moment involve audio processing. This would also allow complete control over audio interruptions since AVQueuePlayer is hardcoded to take over this responsibility.

There are some existing libraries built over the top of AVAudioEngine that could make things easier, which I will have a look at first. For example, AudioKit seems to provide everything we need in terms of audio processing, and there are others (and there's the option of just using the AVAudioEngine API directly), but I will just need to evaluate these options based on how well they support the complete just_audio feature set.

I'd estimate the rewrite of existing features at maybe 1-to-2 months at a minimum, if worked on full time, after which I would expect pitch shifting to be an extra couple of days on top of that.

Given that, I think I may go ahead and release the feature for Android first.

[nt4f04uNd]

i really love the direction you are taking in working on your audio plugins, and how you implement features. these are by far the most mature and will-maintained flutter plugins

[ryanheise]

The Android implementation is now available for testing on the pitch branch.

[ryanheise]

For the first time with this project I'm starting to feel the need for a dev branch (we have a number of features that will need to be tested together before being stable) - I'll do some restructuring.

[afkcodes]

@ryanheise completely agreed with you here, just a suggestion we can also update the readme for the new features in branch. Also eady to support the project, for the good work you are doing.

[ryanheise]

dev is now created. It contains the new pitch feature (Android only so far) and also #300 (buffer configuration options). My immediate goal is to bunch together the features that will involve changes to the platform interface so that I can do a single release (because the release process for changes to the platform interface is usually time consuming.) I will probably also include in this bunch the "skip silence" feature (just Android for now). Once that's all done, I can focus on the iOS implementation.

[ryanheise]

(Copying this comment from another issue to get broader interest)

Is this supported on iOS currently?

The waveform visualizer is implemented on iOS but not pitch. You can track the pitch feature here: #329

There is a big question at this point whether to continue with the current AVQueuePlayer-based implementation or switch to an AVAudioEngine-based implementation. For pitch scaling, I really want to take advantage of AVAudioEngine's built-in features, but that requires a rewrite of the iOS side - see #334 and this is a MUCH bigger project.

I would really like to see an AVAudioEngine-based solution see the light of day, but it will probably not happen if I work on it alone. If anyone would like to help, maybe we can pull it off with some solid open source teamwork. One of the attractive solutions is to use AudioKit which is a library built on top of AVAudioEngine which also provides access to pitch adjustment AND provides a ready-made API for a visualizer and equalizer. That is, it provides us with everything we need - BUT it is written in Swift and so that involves a language change and it means we may need to deal with complaints that old projects don't compile (we'd need to provide extra instructions on how to update their projects to be Swift-compatible).

Would anyone like to help me with this? (Please reply on #334)

[smakarov]

Does it possible to use BASS or Superpowered(best for pitch shifting) or you would like to implement this by yourself?

[ryanheise]

@smakarov this should be relatively straightforward to do once the AVAudioEngine / AudioKit implementation is done. See:

https://audiokit.io/playgrounds/Playback/Time%20Stretching%20and%20Pitch%20Shifting/

So the main effort will go into #334 itself.

[akashrajkn]

Is it also possible to get the pitch of recorded audio?

BTW, this plugin very helpful, thanks for developing it :)

[ghost]

Hi, is there any update for this feature?

[ryanheise]

@gkmuhannad this is already implemented on Android (on the dev branch), iOS will take quite a while longer. I will publish a release including the Android implementation as soon as I update the unit tests.

[ghost]

Great😍.. Thank you so much

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: ryanheise @.> Sent: Sunday, June 20, 2021 5:18:01 PM To: ryanheise/just_audio @.> Cc: Almuhannad Ghaleb @.>; Mention @.> Subject: Re: [ryanheise/just_audio] Pitch adjustment (#329)

@gkmuhannadhttps://github.com/gkmuhannad this is already implemented on Android (on the dev branch), iOS will take quite a while longer. I will publish a release including the Android implementation as soon as I update the unit tests.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ryanheise/just_audio/issues/329#issuecomment-864561959, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATI3EV23IRT7QN47EVWTCTLTTX2BTANCNFSM4YQ52OGQ.

[ryanheise]

The Android implementation is now published in release 0.8.0.

[geekmuhannad]

@ryanheise Hello bro is there any way to save the audio after applying the pitch ?

[ryanheise]

is there any way to save the audio after applying the pitch ?

This plugin is focused on playing audio, not recording or editing it. Those would radically change the way this plugin is designed. But if you can make a good case for including those features, you can make a separate feature request. I think the direction I'm going on iOS may be compatible with this, but the Android side would need a rethink because that's based on ExoPlayer which is also purely focused on playing audio and not recording or editing it.

[ghost]

in our (Social Media App), we are using pitch to change audio after recording, and share it with the other users and I am using your beautiful package for this feature. but unfortunately, the sharing is not supported as you said.

is there any other solution to reach this approach (we are using flutter for android only & native for ios)?

Thanks for your help

---

From: ryanheise @.> Sent: Sunday, July 4, 2021 6:10 AM To: ryanheise/just_audio @.> Cc: Almuhannad Ghaleb @.>; Mention @.> Subject: Re: [ryanheise/just_audio] Pitch adjustment (#329)

is there any way to save the audio after applying the pitch ?

This plugin is focused on playing audio, not recording or editing it. Those would radically change the way this plugin is designed. But if you can make a good case for including those features, you can make a separate feature request. I think the direction I'm going on iOS may be compatible with this, but the Android side would need a rethink because that's based on ExoPlayer which is also purely focused on playing audio and not recording or editing it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ryanheise/just_audio/issues/329#issuecomment-873505910, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATI3EVYDOVGF5L4LSX75ZMLTV7GMBANCNFSM4YQ52OGQ.

[ryanheise]

Maybe editing is possible with ffmpeg but the license isn't ideal for mobile app distribution (maybe you could host a server to do the ffmpeg processing.)

[demberto]

@gkmuhannad I am wondering why nobody mentioned rubber band library, its under GPL tho, but its quite good at what it does and is used in pro audio apps and supports android and ios both. If you are still unsatisfied with that, your only bet is the industry standard zplane (4Tune, Elastique, or Retune) based on what you exactly need. It doesn't mention android support and is proprietary. But its the best, if you want it

[ryanheise]

@demberto thanks for contributing another possible solution, although AudioKit is probably the way to go since it doesn't have the same licensing issues that the GPL has, and AudioKit also solves a large number of other requested features, including the visualiser.

[demberto]

Always happy to help

[SarthakZunroof]

I would like to get change in frequency or change in pitch of an external sound(through microphone) and than animate my visualizer according to the change.If possible give me the solution.

[ryanheise]

@SarthakZunroof please find another plugin that deals with the microphone and make a feature request with that project. It is a bit off topic for the present issue so I will mark both your comment and mine as off topic.

[kekko7072]

Any update on this topic? I need a package to manage pitch and tune, any suggestion?

[ryanheise]

There is some progress being made in #334 on an AVAudioEngine-based implementation. This should make it much easier to implement this functionality on iOS.

Some notes for implementation, these are the classes to look at to adjust the speed and pitch of audio:

• AVAudioUnitVarispeed
• AVAudioUnitTimePitch

[watrindade]

@ryanheise I'm having trouble trying to use it in the version 0.9.20. Is there a way to raise a song pitch just one half step? I mean, a song in C goes to C#?

Thank you the amazing work you have been doing to the community. We very much appreaciate it.

[ryanheise]

I guess you want to know the formula to map a semitone into a pitch adjustment scale. I don't actually know, because the underlying native API just says this scale is "the factor by which pitch will be shifted". An answer to this question will depend on the answer to the same question for native app developers, so I would suggest Googling for that. And if you don't find anything, I would suggest doing a little bit of experimentation and mathematics. E.g. what does a factor of 2x give you? Is it just doubling the frequency? If so, that's a starting point for your calculation.

[akshdeep-singh]

@ryanheise the pitch adjustment doesn't work on android when using just_audio_background? It shows UnimplementedError, but it worked for me before using background plugin.

[ryanheise]

Thanks for reporting, @akshdeep-singh . Actually there are several methods not implemented by just_audio_background:

androidEqualizerBandSetGain
androidEqualizerGetParameters
androidLoudnessEnhancerSetTargetGain
audioEffectSetEnabled
setCanUseNetworkResourcesForLiveStreamingWhilePaused
setPitch
setPreferredPeakBitRate
setSkipSilence

I'll implement these at my next opportunity.

[Jeff09github]

any update for web platform?

[ryanheise]

It's one of the next items on my list, although my main computer has been down so not yet.

[ryanheise]

Sorry I meant to say that fixing setPitch in just_audio_background was my next item. For web, it is not a priority, although contributions are welcome (it will require a different web implementation based on web audio).

@akshdeep-singh I have fixed the unimplemented methods in just_audio_background now. Can you try out the fix in the fix/unimplemented_methods branch and let me know if it is working for you?

[ryanheise]

Regarding fix/unimplemented_methods, I have only tested setPitch although the other methods in the above list should also work now (audio effects, skip silence, etc.)

[akshdeep-singh]

@ryanheise I have verified pitch change, it's working for me. Thanks for your support.

[aamironline]

Thank you very much for creating such a great library. I would like to know the current status of this issue. Could anyone please update? Thanks again.

[ryanheise]

I promise that my comments above truthfully reflect the status, as does the README page.

[alexgrusu]

I can see setPitch is called in setPlatform.

I'm running an app on an iOS simulator and I'm encountering this issue

[ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(-1008, resource unavailable, null, null)

It seems the issue occurs due to the setPitch call.

Is there a way to avoid the setPitch call?

[ryanheise]

Since the platform support is not equal, you can write code like this:

if (Platform.isAndroid) {
    await _player.setPitch(pitch);
}

If you're also cross compiling to web, then you can do this:

if (!kIsWeb && Platform.isAndroid) {
    await _player.setPitch(pitch);
}

[alexgrusu]

Since the platform support is not equal, you can write code like this:

if (Platform.isAndroid) {
    await _player.setPitch(pitch);
}

If you're also cross compiling to web, then you can do this:

if (!kIsWeb && Platform.isAndroid) {
    await _player.setPitch(pitch);
}

I'm not calling setPitch in my code. setPitch is called by setPaltform.

[ryanheise]

There is the following code in setPlatform:

        try {
          await platform.setPitch(SetPitchRequest(pitch: pitch));
        } catch (e) {
          // setPitch not supported on this platform.
        }

So here the exception is handled. Are you sure that this is the call site that is causing the issue? Maybe you're actually using a different plugin such as just_audio_background? Can you reproduce this with one of the official example project, and if so, can you tell me which one and how to reproduce it?

[alexgrusu]

There is the following code in setPlatform:

        try {
          await platform.setPitch(SetPitchRequest(pitch: pitch));
        } catch (e) {
          // setPitch not supported on this platform.
        }

So here the exception is handled. Are you sure that this is the call site that is causing the issue? Maybe you're actually using a different plugin such as just_audio_background? Can you reproduce this with one of the official example project, and if so, can you tell me which one and how to reproduce it?

I'm using just_audio only at the moment.

I tried the official example and I modified the line where the audio source is set to

await _player.setUrl();

and I'm encountering the same error

flutter: Error loading audio source: (-1008) resource unavailable

There might be an issue on my side because the actual URL doesn't contain the file extension and I added the extension to the URL (I mean in the Flutter app)

Is there a way to specify the file extension?

LE: because the URL is something like this: https://erxample/path/file

I added m4a to the end https://erxample/path/file.m4a. I know on iOS the extension is required

[ryanheise]

@alexgrusu Would you mind opening a new bug report? I don't think this issue is specifically related to the setPitch implementation, and if there is a bug somewhere in the setPlatform code, then I would like to investigate that separately from this issue about the setPitch feature.

[alexgrusu]

@alexgrusu Would you mind opening a new bug report? I don't think this issue is specifically related to the setPitch implementation, and if there is a bug somewhere in the setPlatform code, then I would like to investigate that separately from this issue about the setPitch feature.

Sounds good, I will open a different bug.

Might be my approach wrong? I mean, setting the m4a extension manually (the extension is not provided by the URL), could lead to an exception? Is there a way of setting the extension of the file when the link doesn't contain the extension?

[araa12]

any update of pitch change on iOS?