search

ardaku / wavy

Asynchronous cross-platform real-time audio recording & playback.
https://docs.rs/crate/wavy/latest
Apache License 2.0
83 stars 4 forks source link

Default microphone seems not to record #32

Open beckend opened 3 years ago

beckend commented 3 years ago 
use anyhow::Result;
use std::thread::sleep;
use std::time::Duration;
// This example records audio for 5 seconds and writes to a raw PCM file.

use fon::{mono::Mono32, Audio, Frame};
use wavy::{Microphone, MicrophoneStream};

/// An event handled by the event loop.
enum Event<'a> {
  /// Microphone has recorded some audio.
  Record(MicrophoneStream<'a, Mono32>),
}

/// Shared state between tasks on the thread.
struct State {
  /// Temporary buffer for holding real-time audio samples.
  buffer: Audio<Mono32>,
}

impl State {
  /// Event loop.  Return false to stop program.
  fn event(&mut self, event: Event<'_>) {
    match event {
      Event::Record(microphone) => {
        println!("Recording");
        self.buffer.extend(microphone);
        if self.buffer.len() >= 48_000 * 10 {
          write_pcm(&self.buffer);
        }
      }
    }
  }
}

/// Save a Raw PCM File from an audio buffer.
fn write_pcm(buffer: &Audio<Mono32>) {
  let mut pcm: Vec<u8> = Vec::new();
  for frame in buffer.iter() {
    let sample: f32 = frame.channels()[0].into();
    pcm.extend(sample.to_le_bytes().iter());
  }

  dbg!(&pcm);
}

async fn wait() -> Result<()> {
  sleep(Duration::from_millis(2000));

  Ok(())
}

#[tokio::main]
async fn main() -> Result<()> {
  let mut state = State {
    buffer: Audio::with_silence(48_000, 0),
  };
  let mut microphone = Microphone::default();

  state.event(Event::Record(microphone.record().await));

  wait().await?;

  dbg!("finished");

  Ok(())
}

It logs "recording" once, but never reaches to the limit of the buffers to write.

AldaronLau commented 3 years ago

@beckend Thanks, I'll look into why this might be happening.

AnkushMalaker commented 1 year ago

I can observe the same issue on a fresh install of Ubuntu 22.10. It works on another system running Pop!_OS.

I tested the mic and speaker in other applications and they work without a hitch. Running sudo arecord -l also displays my mic. I also tried connecting a second mic, which shows up using the above command and also runs on all other apps but doesn't with wavy.

I tried the "play" example and that doesn't seem to work either using the default speakers.

I already wrote a bunch of code using wavy on my primary system so I hope I can fix this so I won't have to rewrite that to run it on my second system. I can provide any details necessary to debug this. Thanks. @AldaronLau

EDIT: For some additional context

  1. I used Microphone::query() according to this comment and get the following panic:

    ALSA lib pcm_direct.c:1643:(snd1_pcm_direct_initialize_secondary_slave) unable to mmap channels
ALSA lib pcm_dsnoop.c:632:(snd_pcm_dsnoop_open) unable to initialize slave
ALSA lib pcm_dsnoop.c:566:(snd_pcm_dsnoop_open) unable to open slave
thread 'main' panicked at 'Microphone doesn't exist', examples/play.rs:37:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

  2. I used cpal examples to play and record and both work as expected.

It seems that the code isn't going past the Pending stage for both record and play examples.

AldaronLau commented 1 year ago

Thanks, @AnkushMalaker! I haven't had as much time as I would like to work on Wavy recently, but I will use some of my holiday time off to look at this, and updating Wavy. I think this might be related to the case where Wavy is not able to get the requested format of F32LE from ALSA, and doesn't properly handle fallbacks. Can you provide a list of the supported I16/F32/etc., interleaved/non-interleaved, and number of channels settings for the default speakers and microphones of both systems?

AnkushMalaker commented 1 year ago

Sure! Where/how can I get this info? @AldaronLau

AldaronLau commented 1 year ago

@AnkushMalaker you can get it by running:

arecord --dump-hw-params default

Which on my machine, produces

Warning: Some sources (like microphones) may produce inaudible results
         with 8-bit sampling. Use '-f' argument to increase resolution
         e.g. '-f S16_LE'.
Recording WAVE 'default' : Unsigned 8 bit, Rate 8000 Hz, Mono
HW Params of device "(null)":
--------------------
ACCESS:  MMAP_INTERLEAVED MMAP_NONINTERLEAVED RW_INTERLEAVED RW_NONINTERLEAVED
FORMAT:  U8 S16_LE S24_LE S32_LE FLOAT_LE S24_3LE S24_3BE
SUBFORMAT:  STD
SAMPLE_BITS: [8 32]
FRAME_BITS: [8 2048]
CHANNELS: [1 64]
RATE: [1 384000]
PERIOD_TIME: (0 4294967295)
PERIOD_SIZE: (0 2097152]
PERIOD_BYTES: [128 2097152]
PERIODS: [2 1024]
BUFFER_TIME: (2 4294967295]
BUFFER_SIZE: [1 4194304]
BUFFER_BYTES: [256 4194304]
TICK_TIME: ALL
--------------------

Then,

aplay --dump-hw-params default

Which on my machine, produces:

Playing WAVE 'default' : Unsigned 8 bit, Rate 8000 Hz, Mono
HW Params of device "(null)":
--------------------
ACCESS:  MMAP_INTERLEAVED MMAP_NONINTERLEAVED RW_INTERLEAVED RW_NONINTERLEAVED
FORMAT:  U8 S16_LE S24_LE S32_LE FLOAT_LE S24_3LE S24_3BE
SUBFORMAT:  STD
SAMPLE_BITS: [8 32]
FRAME_BITS: [8 2048]
CHANNELS: [1 64]
RATE: [1 384000]
PERIOD_TIME: (0 4294967295)
PERIOD_SIZE: (0 2097152]
PERIOD_BYTES: [128 2097152]
PERIODS: [2 1024]
BUFFER_TIME: (2 4294967295]
BUFFER_SIZE: [1 4194304]
BUFFER_BYTES: [256 4194304]
TICK_TIME: ALL
--------------------
AnkushMalaker commented 1 year ago

Thanks. Here it is. @AldaronLau On the machine where its NOT working:

$arecord --dump-hw-params default

Warning: Some sources (like microphones) may produce inaudiable results
         with 8-bit sampling. Use '-f' argument to increase resolution
         e.g. '-f S16_LE'.
Recording WAVE 'default' : Unsigned 8 bit, Rate 8000 Hz, Mono
HW Params of device "default":
--------------------
ACCESS:  MMAP_INTERLEAVED MMAP_NONINTERLEAVED MMAP_COMPLEX RW_INTERLEAVED RW_NONINTERLEAVED
FORMAT:  S8 U8 S16_LE S16_BE U16_LE U16_BE S24_LE S24_BE U24_LE U24_BE S32_LE S32_BE U32_LE U32_BE FLOAT_LE FLOAT_BE FLOAT64_LE FLOAT64_BE MU_LAW A_LAW IMA_ADPCM S20_LE S20_BE U20_LE U20_BE S24_3LE S24_3BE U24_3LE U24_3BE S20_3LE S20_3BE U20_3LE U20_3BE S18_3LE S18_3BE U18_3LE U18_3BE
SUBFORMAT:  STD
SAMPLE_BITS: [4 64]
FRAME_BITS: [4 640000]
CHANNELS: [1 10000]
RATE: [4000 4294967295)
PERIOD_TIME: (21333 21334)
PERIOD_SIZE: (85 91628833)
PERIOD_BYTES: (42 4294967295)
PERIODS: (0 17247242)
BUFFER_TIME: [1 4294967295]
BUFFER_SIZE: [170 1466015503]
BUFFER_BYTES: [85 4294967295]
TICK_TIME: ALL
--------------------
$aplay --dump-hw-params default

Playing WAVE 'default' : Unsigned 8 bit, Rate 8000 Hz, Mono
HW Params of device "default":
--------------------
ACCESS:  MMAP_INTERLEAVED MMAP_NONINTERLEAVED MMAP_COMPLEX RW_INTERLEAVED RW_NONINTERLEAVED
FORMAT:  S8 U8 S16_LE S16_BE U16_LE U16_BE S24_LE S24_BE U24_LE U24_BE S32_LE S32_BE U32_LE U32_BE FLOAT_LE FLOAT_BE FLOAT64_LE FLOAT64_BE MU_LAW A_LAW IMA_ADPCM S20_LE S20_BE U20_LE U20_BE S24_3LE S24_3BE U24_3LE U24_3BE S20_3LE S20_3BE U20_3LE U20_3BE S18_3LE S18_3BE U18_3LE U18_3BE
SUBFORMAT:  STD
SAMPLE_BITS: [4 64]
FRAME_BITS: [4 640000]
CHANNELS: [1 10000]
RATE: [4000 4294967295)
PERIOD_TIME: (21333 21334)
PERIOD_SIZE: (85 91628833)
PERIOD_BYTES: (42 4294967295)
PERIODS: (0 17247242)
BUFFER_TIME: [1 4294967295]
BUFFER_SIZE: [170 1466015503]
BUFFER_BYTES: [85 4294967295]
TICK_TIME: ALL
--------------------

On the system where it is working:

$ arecord --dump-hw-params default
Warning: Some sources (like microphones) may produce inaudible results
         with 8-bit sampling. Use '-f' argument to increase resolution
         e.g. '-f S16_LE'.
Recording WAVE 'default' : Unsigned 8 bit, Rate 8000 Hz, Mono
HW Params of device "default":
--------------------
ACCESS:  RW_INTERLEAVED
FORMAT:  U8 S16_LE S16_BE S24_LE S24_BE S32_LE S32_BE FLOAT_LE FLOAT_BE MU_LAW A_LAW S24_3LE S24_3BE
SUBFORMAT:  STD
SAMPLE_BITS: [8 32]
FRAME_BITS: [8 1024]
CHANNELS: [1 32]
RATE: [1 384000]
PERIOD_TIME: (2 4294967295)
PERIOD_SIZE: [1 1398102)
PERIOD_BYTES: [128 1398102)
PERIODS: [3 1024]
BUFFER_TIME: (7 4294967295]
BUFFER_SIZE: [3 4194304]
BUFFER_BYTES: [384 4194304]
TICK_TIME: ALL
--------------------
$ aplay --dump-hw-params default
Playing WAVE 'default' : Unsigned 8 bit, Rate 8000 Hz, Mono
HW Params of device "default":
--------------------
ACCESS:  RW_INTERLEAVED
FORMAT:  U8 S16_LE S16_BE S24_LE S24_BE S32_LE S32_BE FLOAT_LE FLOAT_BE MU_LAW A_LAW S24_3LE S24_3BE
SUBFORMAT:  STD
SAMPLE_BITS: [8 32]
FRAME_BITS: [8 1024]
CHANNELS: [1 32]
RATE: [1 384000]
PERIOD_TIME: (2 4294967295)
PERIOD_SIZE: [1 1398102)
PERIOD_BYTES: [128 1398102)
PERIODS: [3 1024]
BUFFER_TIME: (7 4294967295]
BUFFER_SIZE: [3 4194304]
BUFFER_BYTES: [384 4194304]
TICK_TIME: ALL
--------------------

Do note that the audio recording and playing using above command worked on both systems, ie, I could hear the playback in second command of what I recorded in first command.

AnkushMalaker commented 1 year ago

@AldaronLau Hope you had great holidays. Following up on this, is there an update or workaround I can do based on our previous discussion or should I wait for a fix from you and use a different library in the meantime?

AldaronLau commented 1 year ago

@AnkushMalaker I wasn't able to put aside as much time as I would like to look at this. But, it looks like the issue is not what I thought it was. I now suspect it has to do with the something in the poll() implementation. A timeout followed by re-polling the future might fix the issue as a hack-y work around (in the past, there's been similar bugs with this library, just might have missed an edge case in the last fix). I think once I re-implement the poll() method to receive from a channel (a planned change) it may fix the issue.