Open hamoid opened 5 years ago
I haven't run the program yet, but my guess would be that the audio from the mixer is somehow not being read fast enough so it backs up over time.
It's also possible this is just a problem on Linux in particular. I just dug around in the code again to remind myself how the AudioInput works:
TargetDataLine
with a buffer size of 4x of what you request (to prevent buffer underrun). this is a JavaSound object that allows for reading audio from the inputSourceDataLine
with a buffer size of 4x what you request. this is a JavaSound output, which is what makes the monitoring feature of AudioInput
possible. AudioInput
class glues these two together by telling the output object to continuously read from the input object and that mix buffer is updated every time a new buffer is readThis should work OK, but I came across a comment I left for myself in the code that gets the SourceDataLine
about latency just being bad in Linux: https://github.com/ddf/Minim/blob/d5497221c4f55d7202d9166f2c3e77b7d148e956/src/main/java/ddf/minim/javasound/JSMinim.java#L818
All that said, I worked on a project over the summer that also incorporated monitoring the current system audio in order to create reactive visuals. We were working on Windows, essentially through FMOD, and also ran into latency issues there. It wasn't as long as 20 seconds, but it was long enough to look incorrect. Basically I think the only way around this is to somehow mute the audio that is going directly from the browser or wherever and enable monitoring on your AudioInput so that you hear exactly what your program is reading.
You might also try to use a different Linux audio system? I'm not up on what's currently shipping with Linux, but I think I've read in the past that Pulse Audio generally has better latency than ALSA?
Another thing you could try, is to use getInputStream
instead of getLineIn
. This will give you an AudioStream
(https://github.com/ddf/Minim/blob/master/src/main/java/ddf/minim/spi/AudioStream.java) object that you must call read
on to get data out of it. It won't automatically pull. So in draw
you'd read from the AudioStream
until it read fewer samples than your bufferSize
. Best would be to create MultiChannelBuffer
(https://github.com/ddf/Minim/blob/master/src/main/java/ddf/minim/MultiChannelBuffer.java) with the same bufferSize
that you use for the AudioStream
and use the read
method that takes a MultiChannelBuffer
and returns number of samples read. You'd have to decide whether to run the FFT on every buffer or accumulate it somehow or whatever. Definitely it's more complex code and is why AudioInput
exists in the first place.
This might help you determine whether the latency is coming from the read loop of AudioInput
or if it is latency in putting audio data into the Java's TargetDataLine
on the Linux side.
Came through from the Processing forum post on this. In my experience Linux has by far the best latency performance of all the OS JavaSound implementations. I'm intrigued why we get such different results. Will try and take a look sometime.
Thank you for the detailed answer!!! I'll do tests and report back :)
@hamoid Curious if you have anything to report about all this.
Hey guys, I had the same problem on Elementary Juno (Ubuntu 18.04 LTS), Processing 3.5.3. I implemented @ddf's recommendation and reading from an inputStream seems to do the trick for me. This is my QaD-code:
import ddf.minim.*;
import ddf.minim.spi.*;
Minim minim;
AudioStream in;
MultiChannelBuffer sampleBuffer;
void setup()
{
size(512, 200, P3D);
minim = new Minim(this);
in = minim.getInputStream(2,1024,44100.0,16);
in.open();
sampleBuffer = new MultiChannelBuffer(1024, 2);
}
void draw()
{
background(0);
stroke(255);
// draw the waveforms so we can see what we are monitoring
in.read(sampleBuffer);
for(int i = 0; i < 1023; i++)
{
line( i, 50 + sampleBuffer.getSample(0,i)*50, i+1, 50 + sampleBuffer.getSample(0,i+1)*50 );
line( i, 150 + sampleBuffer.getSample(1,i)*50, i+1, 150 + sampleBuffer.getSample(1,i+1)*50 );
}
}
void stop() {
in.close();
}
Could someone verify this on their system? update: it didn't go as well as I expected. Findings:
audio
usergroup for high priority.pacmd list-sources
to check some latency metrics, my latency metrics were all over the place.Right now I reverted to using an audiosplitter (hardware) and rewire one output to the mic input and use that for audio analysis. It's noisy on the low-end spectrum but at least it's fast.
I just added a Minim example to OPENRNDR and remembered this issue.
I tried my code above and also the one from @rvorias but I couldn't start diagnosing things because I'm getting the microphone input instead of my system audio, which was my original plan. I tried the different devices returned by AudioSystem.getMixerInfo();
(on Ubuntu).
At least the microphone input seems to have low latency :)
I'm trying to do FFT on live audio from the mixer, so I can feed data into sketches based on whatever music I happen to be playing. The issue I'm having is latency: between 2 and 20 seconds. And it seems to drift, so I believe it increases while the program runs.
Could it be something related to mismatching bitrates or buffer sizes?
I also asked about it on the forum.