intel / openvino-plugins-ai-audacity

A set of AI-enabled effects, generators, and analyzers for Audacity®.
GNU General Public License v3.0
1.01k stars 64 forks source link

Testing OpenVINO Noise Suppression with Audacity 3.4.2 #5

Closed petersampsonaudacity closed 8 months ago

petersampsonaudacity commented 11 months ago

I tested Noise Suppression on a commercially released CD - the same track I used in #4 This is Michelle Holding and Bonz with Michelle singing in Lancashire dialect and Bonz accompanying her on banjo, so no "noise" to speak or really.

Here the result was disappointing. I ended up with what sounded like a poor instrument removal - vocals with occasional percussive thumps from the banjo but no banjo tune.

Now some may say that removing banjo is no bad thing ;-) - but for the purposes of this test ...

petersampsonaudacity commented 11 months ago

So then I tried on on Sgt Pepper track-1 and it took out almost everything (I used default settings): image

RyanMetcalfeInt8 commented 11 months ago

@petersampsonaudacity -- Thank you! It's a bit unclear from the documentation (sorry about that), but this noise suppression model was designed to remove background noise from spoken audio (as opposed to removing generic 'noise' from, say, musical tracks).

So in this case, it will treat all non-vocals as 'noise' -- so I think your results make sense.

With that said, it would be really nice to support a more generic 'noise removal' effect, which is what I think you may have expected from this to begin with. I'll keep an eye out for emerging open source AI models that we could leverage for this (as this is what it comes down to).

If anyone has spotted anything that we could leverage here, please point it out.

Thanks!

petersampsonaudacity commented 11 months ago

@RyanMetcalfeInt8 It night be useful if the user could also (optionally) see a track with the noise that is taken out "suppressed".

I say this based on my experience with Brian Davies' excellent ClickRepair which I used extensively when I was transcribing my LPs where being able to see and hear to noise/clicks that would be removed enables one to fine-tune the parameter settings image So here you can choose to hear a) the noise b) the noisy input signal c) the de-noised output signal and flit between them as processing takes place - most useful.

petersampsonaudacity commented 11 months ago

So in this case, it will treat all non-vocals as 'noise' -- so I think your results make sense.

But with Sgt Pepper it also removed 95% or more of the vocals

RyanMetcalfeInt8 commented 11 months ago

@RyanMetcalfeInt8 It night be useful if the user could also (optionally) see a track with the noise that is taken out "suppressed".

I say this based on my experience with Brian Davies' excellent ClickRepair which I used extensively when I was transcribing my LPs where being able to see and hear to noise/clicks that would be removed enables one to fine-tune the parameter settings image So here you can choose to hear a) the noise b) the noisy input signal c) the de-noised output signal and flit between them as processing takes place - most useful.

Interesting - This would be definitely be something useful, I agree. Thanks for sharing.

petersampsonaudacity commented 11 months ago

It's a bit unclear from the documentation (sorry about that), but this noise suppression model was designed to remove background noise from spoken audio (as opposed to removing generic 'noise' from, say, musical tracks).

@RyanMetcalfeInt8

In which case you should probably talk to Paul Licameli who is one of the Muse Audacity developers (and was, like me, a member of the former Audacity Team before Muse took over the project).

Paul is interested in VoiceOver and runs a Facebook group for Audacity VoiceOver users: https://www.facebook.com/groups/562223697221079

You can find Paul on the Muse Discord or on their GitHub

RyanMetcalfeInt8 commented 11 months ago

So in this case, it will treat all non-vocals as 'noise' -- so I think your results make sense.

But with Sgt Pepper it also removed 95% or more of the vocals

Hmm, I'll experiment with it some more. It could be that the model was trained on primarily 'spoken' audio.. I'm not quite sure... I think in general, more cutting edge / sophisticated noise removal models would make a difference here.

RyanMetcalfeInt8 commented 11 months ago

It's a bit unclear from the documentation (sorry about that), but this noise suppression model was designed to remove background noise from spoken audio (as opposed to removing generic 'noise' from, say, musical tracks).

@RyanMetcalfeInt8

In which case you should probably talk to Paul Licameli who is one of the Muse Audacity developers (and was, like me, a member of the former Audacity Team before Muse took over the project).

Paul is interested in VoiceOver and runs a Facebook group for Audacity VoiceOver users: https://www.facebook.com/groups/562223697221079

You can find Paul on the Muse Discord or on their GitHub

Cool, thanks! I've actually been working quite closely with Paul over the past few months. He has been extremely helpful in answering questions & actually pushing changes to Audacity repo itself to help support this.