mikeoliphant / AudioPlugSharp

Easily create VST (VST3) audio plugins in C# .NET
MIT License
177 stars 21 forks source link

A Clarification #8

Closed MOAMaster closed 2 years ago

MOAMaster commented 2 years ago

I am a bit confused about how the host and the plugin interact.

How can a plug-in detect that real data is coming through because the operator has pushed the start button?

From what I can tell using message boxes, when the operator loads the plug in into the host, a series of function calls are made to the plugin which include Initialize, InitializeProcessing, and Start. Then, regular calls to Process are made -- even though the user has not clicked on play in the host.

I have an algorithm that I only want to run in the presence of actual data. If this algorithm runs without real data, then the algorithm crashes the application.

I tried checking if the inputSamples.Length is > 0. Well, that value is greater than zero although there is no real data to process.

So, how do I detect that there is real data in the input buffer? I thought that Start or InitializeProcessing would be called just after the operator clicked on Play or that no Process calls would be made if the operator had not clicked on Play. But, apparently, all of those are invoked at the time the plugin is loaded.

So, How can a plug-in detect that real data is coming through because the operator has pushed the start button? Any ideas?

You can't check for all zeros in the input buffer because that would also indicate silence.

Thanks.

Bruce

MOAMaster commented 2 years ago

I think I see what is missing. I went into the Steinburg spec. Here is what I found:

tresult PLUGIN_API PlugProcessor::process (Vst::ProcessData& data) { // first they read any parameter changes used for processing the audio, then --

if (data.numInputs == 0 || data.numSamples == 0) return kResultOk; }

So, the host is sending these two fields to allow you to skip processing if no real data is passed.

Your wrapper called AudioIOPort needs to forward these two key values to the plugin so we can avoid any real processing.

Looking at the length of the input buffers does not work. I checked. An empty buffer is being sent, in my case with 980 or so items.

I think that having these kind of guards is critical to reliable performance. My algorithm is blowing up because it's trying to execute on empty arrays.

I'll try coding in an exception handler, that might solve the problem. But would really like to follow the spec.

Thanks.

Bruce

It looks like the host is sending these two fields in

mikeoliphant commented 2 years ago

The AudioIOPort gives you the number of samples in CurrentBufferSize.

MOAMaster commented 2 years ago

CurrentBufferSize is the size of the buffer.

Is that 0 when the buffer does not contain valid info?

Because the host immediately begins sending invalid buffers.

They are used as a signal that allows the plugin to update

the values of any parameters.

But, they don’t have valid data in them.

That is why they had those two other variables to use as guards.

I’ll do a test and see if the CurrentBufferSize protects you from going into processing

with invalid data. If it doesn’t, I’ll get blowups again.

Please don’t close this item until I do the test.

Thanks.

Bruce

From: Mike Oliphant @.> Sent: Monday, January 10, 2022 9:08 AM To: mikeoliphant/AudioPlugSharp @.> Cc: MOAMaster @.>; Author @.> Subject: Re: [mikeoliphant/AudioPlugSharp] A Clarification (Issue #8)

The AudioIOPort gives you the number of samples in CurrentBufferSize.

— Reply to this email directly, view it on GitHub https://github.com/mikeoliphant/AudioPlugSharp/issues/8#issuecomment-1009138878 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2KTWT4QGXWEHTUX3UX3RDUVMHATANCNFSM5LRNJKCA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AE2KTWVHJDO44Q3GS5UNYMLUVMHATA5CNFSM5LRNJKCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHQTDZPQ.gif Message ID: @. @.> >

mikeoliphant commented 2 years ago

CurrentBufferSize should be exactly what the host sent as the number of samples to process.

MOAMaster commented 2 years ago

Mike,

I did a test. CurrentBufferSize does NOT work as a guard.

Let me first explain a bit how the host interacts with the plugin.

I found this out by putting Message Boxes into each of the overridden methods In the plugin.

The plugin is loaded.

At the time the plugin is loaded, the following methods are called:

Initialize

InitializeProcessing Start

Then, the host begins making calls to Process.

These calls to Process continue even though the user has not started playing.

The calls include input buffers that are full size with garbage.

Calls to stereoInput.GetBufferSize return a non-zero number even

though sound has not started playing. That is because the host

calls Process and allocates a full buffer. The full size buffer does NOT

include valid data because the user has not started playing.

In the original spec, they make this clear because they have those

two guard variables for you to check if the input buffer contains real data.

When I don’t have the two guard variables, my data processing algorithms

are trying to process junk and blow up. So, you need a means to bypass

any actual buffer processing if no real data is provided.

The two guard variables in the original spec need to be provided, NOT the size.

Because you always get a size > 0 even though you aren’t playing any music.

My test is simple.

Do not implement a guard. Load the plugin into a host. Do NOT start any music. Start

manipulating the knob. My plugin tries to process the buffer through the algorithm.

Bam, I get a blowup.

When I put an exception handler around the algorithm, the catch block simply

transfers the input to the output. I can manipulate the knob all day without a blowup.

That further confirms that the calls to Process before any music is being played

return a nonzero Size.

You can use a message box to display the results of CurrentBufferSize and see that

you get a non-zero value even though you haven’t started playing any music.

I don’t know how to make this any more clear.

Your simple examples don’t have this issue because you are multiplying the

zero buffer values by a constant.

Others are going to have this problem.

Please, either make those guard variables available or update your

simple examples to put a try/catch block around the data processing.

I taught a lot of courses over the years. Many classes were how to

write programs. I spent a full hour explaining the conditions for use

and for avoiding exceptions. This is a routine situation which should

be handled by guard variables and not by exceptions.

Thanks for listening.

Bruce

From: Mike Oliphant @.> Sent: Monday, January 10, 2022 9:46 AM To: mikeoliphant/AudioPlugSharp @.> Cc: MOAMaster @.>; Author @.> Subject: Re: [mikeoliphant/AudioPlugSharp] A Clarification (Issue #8)

CurrentBufferSize should be exactly what the host sent as the number of samples to process.

— Reply to this email directly, view it on GitHub https://github.com/mikeoliphant/AudioPlugSharp/issues/8#issuecomment-1009172512 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2KTWX5O6SX6VW4HR22YELUVMLO7ANCNFSM5LRNJKCA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AE2KTWVS63P4YBMMSCQCEOLUVMLO7A5CNFSM5LRNJKCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHQTMAIA.gif Message ID: @. @.> >

mikeoliphant commented 2 years ago

What host are you using?

MOAMaster commented 2 years ago

Cakewalk and waveform.

Also i wrote instructions to set up plugin projects in an indept vs solution. A bit tricky but i can send to you to post.

If you create plugin projects in your solution desktop wants to remove your projects when you update solution

Sent from my iPhone

On Jan 10, 2022, at 12:22 PM, Mike Oliphant @.***> wrote:

 What host are you using?

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.

mikeoliphant commented 2 years ago

When and how the VST process function is called is entirely up to the host. It is common for a host to continually call Process even when playback is not happening.

I installed Cakewalk, and verified that it is indeed calling Process even when playback is stopped. The buffers it sends are filled with silence.

MOAMaster commented 2 years ago

The problem is that i cant use silence as an indicator of validity to apply the algorithm because an audio file may contain silence.

I dont understand your reluctance to implement those guard values as the steinberg spec and sample programs show their use.

I wasted several days identifying this issue and a workaround. Everyone who uses your framework will have this issue. You need to find a way for the plugin to know that real data is included.

Thanks

Sent from my iPhone

On Jan 10, 2022, at 1:22 PM, Mike Oliphant @.***> wrote:

 When and how the VST process function is called is entirely up to the host. It is common for a host to continually call Process even when playback is not happening.

I installed Cakewalk, and verified that it is indeed calling Process even when playback is stopped. The buffers it sends are filled with silence.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.

mikeoliphant commented 2 years ago

There is nothing to work around here. The host is making perfectly valid process calls that should not be skipped. I am not aware of any way to distinguish them from the calls made during playback.

mikeoliphant commented 2 years ago

Your core issue seems to be that you need to know if playback is active or not. I'm not sure that is possible.

Reaper (the DAW I use) has an option to turn off plugin processing when playback is not active. Maybe Cakewalk has a similar option?

MOAMaster commented 2 years ago

According to the examples in the steinberg manual , those 2 variables seem to do that.

Oh well i tried. Disappointed.

Sent from my iPhone

On Jan 10, 2022, at 1:58 PM, Mike Oliphant @.***> wrote:

 There is nothing to work around here. The host is making perfectly valid process calls that should not be skipped. I am not aware of any way to distinguish them from the calls made during playback.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.

MOAMaster commented 2 years ago

That is a good suggestion. I ‘ll check cakewalk

Sent from my iPhone

On Jan 10, 2022, at 2:03 PM, Mike Oliphant @.***> wrote:

 Your core issue seems to be that you need to know if playback is active or not. I'm not sure that is possible.

Reaper (the DAW I use) has an option to turn off plugin processing when playback is not active. Maybe Cakewalk has a similar option?

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.

mikeoliphant commented 2 years ago

According to the examples in the steinberg manual , those 2 variables seem to do that. Oh well i tried. Disappointed.

If by "those 2 variables" you mean "numInputs" and "numSamples", the AudioPlugSharp code is paying attention to them. It doesn't call the C# Process method if "numInputs" and "numOutputs" are zero, and it passes on the value of "numSamples" in CurrentBufferSize.

The problem is that in the case you are dealing with, "numInputs", "numOutputs" and "numSamples" are not zero.

MOAMaster commented 2 years ago

One last question.

In the Steinberg spec, they check:

          if (data.numInputs == 0 || data.numOutputs == 0)

In vstbypassprocessor.h, that is what you use in the process method.

But in AudioPlugSharpProcessor, in the process method you use:

           if (data.numInputs == 0 && data.numOutputs == 0)

That is clearly a different condition.

Do you think that changing the && to || might stop the non-audio calls to process in the plugin?

Thanks.

Bruce

From: Mike Oliphant @.> Sent: Monday, January 10, 2022 2:14 PM To: mikeoliphant/AudioPlugSharp @.> Cc: MOAMaster @.>; Author @.> Subject: Re: [mikeoliphant/AudioPlugSharp] A Clarification (Issue #8)

According to the examples in the steinberg manual , those 2 variables seem to do that. Oh well i tried. Disappointed.

If by "those 2 variables" you mean "numInputs" and "numSamples", the AudioPlugSharp code is paying attention to them. It doesn't call the C# Process method if "numInputs" and "numOutputs" are zero, and it passes on the value of "numSamples" in CurrentBufferSize.

The problem is that in the case you are dealing with, "numInputs", "numOutputs" and "numSamples" are not zero.

— Reply to this email directly, view it on GitHub https://github.com/mikeoliphant/AudioPlugSharp/issues/8#issuecomment-1009394903 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2KTWQO7ALJVVCMKV6GRWTUVNK3VANCNFSM5LRNJKCA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AE2KTWTFRRU5IZPEJ27WAL3UVNK3VA5CNFSM5LRNJKCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHQVCJVY.gif Message ID: @. @.> >

mikeoliphant commented 2 years ago

One last question. In the Steinberg spec, they check: if (data.numInputs == 0 || data.numOutputs == 0) In vstbypassprocessor.h, that is what you use in the process method. But in AudioPlugSharpProcessor, in the process method you use: if (data.numInputs == 0 && data.numOutputs == 0) That is clearly a different condition. Do you think that changing the && to || might stop the non-audio calls to process in the plugin?

Nope. The non-playback calls pass values >0 for both numInputs and numOutputs. I use "&&" because it is reasonable that a plugin could have either no inputs (an instrument) or no outputs (a visualizer), but not both.

MOAMaster commented 2 years ago

it is reasonable that a plugin could have either no inputs (an instrument) or no outputs (a visualizer), but not both.

Yep, but && requires both not one or the other.

One or the other uses ||.

Bruce

From: Mike Oliphant @.> Sent: Monday, January 10, 2022 4:53 PM To: mikeoliphant/AudioPlugSharp @.> Cc: MOAMaster @.>; Author @.> Subject: Re: [mikeoliphant/AudioPlugSharp] A Clarification (Issue #8)

One last question. In the Steinberg spec, they check: if (data.numInputs == 0 || data.numOutputs == 0) In vstbypassprocessor.h, that is what you use in the process method. But in AudioPlugSharpProcessor, in the process method you use: if (data.numInputs == 0 && data.numOutputs == 0) That is clearly a different condition. Do you think that changing the && to || might stop the non-audio calls to process in the plugin?

Nope. The non-playback calls pass values >0 for both numInputs and numOutputs. I use "&&" because it is reasonable that a plugin could have either no inputs (an instrument) or no outputs (a visualizer), but not both.

— Reply to this email directly, view it on GitHub https://github.com/mikeoliphant/AudioPlugSharp/issues/8#issuecomment-1009494413 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2KTWTJ7CL43IGCH6SRJ2TUVN5P5ANCNFSM5LRNJKCA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AE2KTWXT4K33E6UVMAWHKDDUVN5P5A5CNFSM5LRNJKCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHQV2TDI.gif Message ID: @. @.> >

mikeoliphant commented 2 years ago

Bruce - I'm quite familiar with how logical operators work. The logic is working as I described it - it bypasses processing if both numInputs and numOutputs are zero.

Regardless, it make no difference in the case you are encountering as both are non-zero.

MOAMaster commented 2 years ago

As you say, there is no way to know if you have junk or real audio.

I just left the exceptions in there.

I also did another test. I inserted into AudioPlugSharpProcessor:

if (data.numSamples == 0) return kResultOk;

I hoped that would perhaps stop the call to process in the plugin.

Nope, no such luck.

As you said, there does not seem to be any way to know if the user

has started sending audio.

I just left the try/catch block in place.

The problem goes away.

Thanks again for your patience.

Bruce

From: Mike Oliphant @.> Sent: Monday, January 10, 2022 5:33 PM To: mikeoliphant/AudioPlugSharp @.> Cc: MOAMaster @.>; Author @.> Subject: Re: [mikeoliphant/AudioPlugSharp] A Clarification (Issue #8)

Bruce - I'm quite familiar with how logical operators work. The logic is working as I described it - it bypasses processing if both numInputs and numOutputs are zero.

Regardless, it make no difference in the case you are encountering as both are non-zero.

— Reply to this email directly, view it on GitHub https://github.com/mikeoliphant/AudioPlugSharp/issues/8#issuecomment-1009519852 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2KTWULHFCHGCGRGBGQJV3UVOCEBANCNFSM5LRNJKCA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AE2KTWUK2SKECCBTXLUF7K3UVOCEBA5CNFSM5LRNJKCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHQWAZ3A.gif Message ID: @. @.> >

MOAMaster commented 2 years ago

Thanks again for your patience, the interaction, and the well designed wrapper classes.

Bruce

From: Mike Oliphant @.> Sent: Monday, January 10, 2022 7:06 PM To: mikeoliphant/AudioPlugSharp @.> Cc: MOAMaster @.>; Author @.> Subject: Re: [mikeoliphant/AudioPlugSharp] A Clarification (Issue #8)

Closed #8 https://github.com/mikeoliphant/AudioPlugSharp/issues/8 .

— Reply to this email directly, view it on GitHub https://github.com/mikeoliphant/AudioPlugSharp/issues/8#event-5870137377 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2KTWS7LYWOQRR5IMFB2STUVONBRANCNFSM5LRNJKCA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AE2KTWWE4FVQRQKOFSAIK23UVONBRA5CNFSM5LRNJKCKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGPAAAAAAK54MYCC.gif Message ID: @. @.> >

mikeoliphant commented 2 years ago

:thumbsup:

MOAMaster commented 2 years ago

Mike,

I want to open a discussion with you about a topic.

It’s not really an issue but I’ll start the discussion.

If you would prefer to have this discussion as an issue,

let me know. I’ll transition to there.

I have written and used WinForms and WPF.

I find WPF very difficult to use.

I need to add text labels, buttons, and sliders to the editor window.

Also, might want to use different knobs.

Very difficult to do this in WPF in the setup with your framework.

So, I want to do a winforms version.

I have created a version that I can mess with that will allow me to experiment.

Wondered if you could make some suggestions as to how I might proceed?

I copied AudioPlugSharpWPF into a project named AudioPlugSharpWPM.

Changed all the name spaces.

I know I will have to delete the xaml files and then somehow replace what they do.

Any other suggestions you can give me about how to proceed.

For instance, right now VS does not recognize EditorWindow although I changed the namespaces.

Any suggestions?

Thanks.

Bruce

From: Mike Oliphant @.> Sent: Tuesday, January 11, 2022 12:45 PM To: mikeoliphant/AudioPlugSharp @.> Cc: MOAMaster @.>; Author @.> Subject: Re: [mikeoliphant/AudioPlugSharp] A Clarification (Issue #8)

👍

— Reply to this email directly, view it on GitHub https://github.com/mikeoliphant/AudioPlugSharp/issues/8#issuecomment-1010345253 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2KTWTMROLOI7YF4B6J2GDUVSJEDANCNFSM5LRNJKCA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AE2KTWUWJ4TH2EWD3K6TS4LUVSJEDA5CNFSM5LRNJKCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHQ4KKJI.gif Message ID: @. @.> >