MA_NODE_FLAG_SILENT is not usable (or i do not know how...)

[santiky]

Hi,

i'm developing an audio mixer with minilib, i've been struggling with the multi-device problem and i have a prototype with two devices working on a ring buffer, a writer node in one device and a data source in another that reads, like suggested in mackron/miniaudio#378.

It works but MA_NODE_FLAG_SILENT is unusable because it duplicates the recursive trigger API in the sound chain, so the sounds are played faster than normal. I have done a vumeter too and it has the same behavior. The only solution i have found is inserting the nodes in the sound chain and pass the sound through but it could be fine having the silent flag working. For send buffers it's fine, but for vumeters i would like to meter the signals without inserting the node into the main sound chain, so the meters can be disabled and do no impact in the audio processing. Am i doing something wrong or have i lost something? i have read every line of doc and issues before asking...

Resuming: if we attach two times the endpoint in one audio chain the recursive read pcm frames trigger is launched two times for each one pcm frames, attaching n times, it accelerates n times.

It would be nice that the SILENT flag does not triggers the recursive call or have a node that can stops the trigger in the chain (a bang in pure data terminology?). In Pure Data there are the "hot and cold" inlets concept, hot inlets launch the audio chain process, cold only make some change settings on node or make some process without refilling the buffers, is it possible a splitter node that only takes the trigger from a single output?

This works fine:

static void ma_writer_node_process_pcm_frames(ma_node* pNode, const float** ppFramesIn, ma_uint32* pFrameCountIn, float** ppFramesOut, ma_uint32* pFrameCountOut)
{
    ma_writer_node* pWriteNode = (ma_writer_node*)pNode;

    MA_ASSERT(pWriteNode != NULL);
    MA_ASSERT(ma_node_get_input_bus_count(&pWriteNode->baseNode) == 2);

    if (*pFrameCountIn > 0) {
        void *pWriteBuffer = NULL;
        ma_pcm_rb_acquire_write(pWriteNode->pBuffer, pFrameCountIn, &pWriteBuffer);
        if (pWriteBuffer != NULL) {
            ma_copy_pcm_frames(pWriteBuffer, ppFramesIn[1], *pFrameCountIn, ma_format_f32, pWriteNode->channels);
            ma_pcm_rb_commit_write(pWriteNode->pBuffer, *pFrameCountIn);
        }
    }
    ma_copy_pcm_frames(ppFramesOut[0], ppFramesIn[0], *pFrameCountOut, ma_format_f32, pWriteNode->channels);
}

static ma_node_vtable g_ma_writer_node_vtable =
{
    ma_writer_node_process_pcm_frames,
    NULL,
    2,
    1,
    0
};

This speed up the audio chain:

static void ma_vumeter_node_process_pcm_frames(ma_node* pNode, const float** ppFramesIn, ma_uint32* pFrameCountIn, float** ppFramesOut, ma_uint32* pFrameCountOut)
{
    ma_vumeter_node* pVumeterNode = (ma_vumeter_node*)pNode;

    MA_ASSERT(pVumeterNode != NULL);
    //MA_ASSERT(ma_node_get_input_bus_count(&pVumeterNode->baseNode) == 1);

    for (uint i = 0; i < *pFrameCountIn; i++) {
        float input = fabsf(ppFramesIn[0][i]);
        pVumeterNode->level += pVumeterNode->alpha * (input - pVumeterNode->level);
    }
    *pFrameCountOut = 0;
    (void)ppFramesOut;
}

static ma_node_vtable g_ma_vumeter_node_vtable =
{
    ma_vumeter_node_process_pcm_frames,
    NULL,
    1,
    1,
    MA_NODE_FLAG_SILENT_OUTPUT
};

I have uploaded working examples in https://github.com/santiky/miniaudio-extras ma_writer_node_example is running fine passing audio ma_vumeter_node_example is accelerating the audio chain.

Congratulations for miniaudio, it's an impressive library and it works faultlessly.

[mackron]

I think you might have hit a limitation in miniaudio's graph system that I didn't anticipate. So this is your setup for the vumeter part?

test.flac --> splitter --> vumeter --> endpoint
                 |                        ^
                 |                        |
                 +------------------------+

What I think might be happening is the endpoint sees that the splitter is attached to it, so it pulls data from the splitter, and then it sees the vumeter is attached to it and then pulls from that. But then the problem is that the vumeter node then pulls from the splitter again which results in too much data being read and it sounding like the sound is accelerating.

I haven't yet compiled your sample code. Is it easy to compile? No external dependencies or anything?

I'm not sure off the top of my head how to address this, but if my suspicion proves to be correct it's probably something I should figure out a solution for.

[khiner]

FWIW, I have exactly this kind of toggle-able vu-meter option for each graph node in my application, and I "solved" this problem by just inserting/removing the vu-meter sub-node from the graph. So each logical "node" in my graphs are either just the main processing node, and optionally a monitor node, panner node, and gain node in series.

  NODE
  -----------------------------------------------------------------------
  |                                                                     |
->|(in_gain?->)(in_monitor?->)[processor](->out_monitor?)(->out_gain?)->|->
  |                                                                     |
  -----------------------------------------------------------------------

Each of the above optional sub-processors gets inserted/removed from the audio graph when enabled/disabled. I know it's not ideal and maybe this doesn't address your use case, but thought I'd share. I remember reading a response from @mackron at one point that inspection callbacks in the graph is a potential future goal for the API (found it - https://github.com/mackron/miniaudio/discussions/674#discussioncomment-5971973). Is that still a possibility at some point?

[mackron]

That callback thing is already in the dev-0.12 branch so that'll definitely be coming. However, that's at the ma_sound level which is higher level than ma_node (ma_sound wraps around ma_node). Not sure if I'll be adding that to the ma_node level. I noticed in that comment you linked to that I even suggested the exact same set up as @santiky mentioned in the original post. I'll need to replicate this and investigate. So you were getting the same symptoms - sped up audio output?

[santiky]

test.flac --> splitter --> vumeter --> endpoint
                 |                        ^
                 |                        |
                 +------------------------+

yes, it is.

I haven't yet compiled your sample code. Is it easy to compile? No external dependencies or anything?

no dependencies, only miniaudio. i'm compiling in ubuntu 22.04 with:

gcc -g ma_writer_node.c ma_vumeter_example.c -lpthread -ldl -lm

I'm not sure off the top of my head how to address this, but if my suspicion proves to be correct it's probably something I should figure out a solution for.

blocking pull requests on all output buses of splitter nodes except the first, i think it would do the trick. Or making some "block" node that stops the pull request, that is what i thought the MA_NODE_SILENT_FLAG were. I do not see any use to splitter node like is now, every node needs to be attached to endpoint to process their data, right? So in each case of use, it will speed up the audio chain. I have taken a look in miniaudio sources but i do now know where to start...

[santiky]

Each of the above optional sub-processors gets inserted/removed from the audio graph when enabled/disabled.

I have a similar architecture, i work with "layers" that copy typical analog desk channels:

source -> splitter2 |-> Filter Bank -|-> splittern |-> writer -> output
                    |                |             |
                    |----------------|             |-> Aux1
                            Bypass                 |-> Aux2

I would like to have one vumeter wich an user could attach in any point of the chain, or at least in the input splitter (that acts as a gain too) and the splittern, that acts as the "aux sends" knobs in analog consoles. I'm trying to avoid attaching and dettaching because i thought that it would cause pops and clicks (for example the filter bypass i'm using the output bus volume instead attaching), can we attach or dettach in live without concerns?

[khiner]

I noticed in that comment you linked to that I even suggested the exact same set up as @santiky mentioned in the original post. I'll need to replicate this and investigate. So you were getting the same symptoms - sped up audio output?

Thanks for checking on this - I ended up opting for dynamically adding/removing monitors from the node graph to completely avoid any buffer copying/processing when not monitoring (including the splitter).

[mackron]

Sorry the unacceptably long delay on this. I have pushed a potential fix for this to the dev branch. Are you able to verify that fix on your end?

[santiky]

I just ran a quick test and it's playing at the correct speed, thanks.