Microphone array documentation

[doumdi]

Could you please add information/documentation on how the microphone array works. Are the 8 channels available as "raw data" in the Raspberry Pi? What is the audio interface (I2S) ?

Thanks!

[eljuguetero]

Hi,

We already implemented a raw 16 bits signed 16000 Hz raw interface over SPI https://github.com/matrix-io/matrix-creator-hal/blob/master/demos/mic_demo.cpp

Better examples will be published the next week.

Best regards,

-- Andrés

[doumdi]

Thanks for the quick answer. This will be perfect for voice applications! Will it be possible to have a higher sampling rate in the future?

[eljuguetero]

Yes, we will improve the sampling rate. In your wish list, what sampling rate do you want?

[doumdi]

48kHz would be great x 8 channels, 16bits. The intended application is audio beamforming. The sampling of the 8 microphones must be synchronous. How big is the audio buffer in the FPGA?

[doumdi]

It seems the SPI communication is clocked at 25MHz. 8 microphones @ 48kHz, 16 bits = 6.144Mbps. That should fit.

[eljuguetero]

2kB right now, but we have enough BRAMs to increase to 8kB.

[doumdi]

8kB should store approximately 10ms of audio @ 48kHz. This could work.

[Flowr-es]

Will this mic array demo also be available in nodejs?

[buildog]

+1 for js documentation for the mic & IR TX/RX

[preygel]

Second the suggestion on a higher sampling rate. I'd love either 44.1 or 48kHz support.

[Or if not, then perhaps a point to the current implementation / any documentation or considerations that would be useful in implementing the higher sample rate oneself.]

[namanigandhi]

How to enable 10ms of audio @ 48kHz, 8 channel recording. Do we need to set any settings on Raspberry Pi.

[ruffsl]

Any update on this? I too would like to achieve x8 channels @ 48kHz, 16bits to use in audio beamforming for sound source localization and tracking.

[alejandrojapkin]

Is there a way to use the Alexa demo you guys created to simply stream the audio coming from the array, through wifi, to another system and for other purposes?

[paul-ml]

what will be the output of each channel separately?

[alejandrojapkin]

As was said earlier, people want to use the mic array to build custom beamforming experiments.

Matrix Creator was a disappointment, it was sold as a dev & test lab kit and it wasn't, rather a rushed & pretentious FPGA toy with a bunch of sensors crammed in for which a vague SDK and poor documentation was provided, mainly focused on accommodating the specific demo applications Matrix showcased to sell this. That is exactly what a dev board is not supposed to be. On top of that, terrible support: just look at how old this ticket is, after some social engineering I was able to reach them and I got this: "we're going to launch another product soon with x,y,z features which should accommodate your requirements." Anyone with capital can go to a chinese factory and commission 10000 FPGAs with lots o' sensors, but selling boards makes it your job to release a capable SDK, properly document and support for it on every angle, neither things Matrix did. Otherwise we'd all be making money stamping our band on chinese electronics.

[ruffsl]

As was said earlier, people want to use the mic array to build custom beamforming experiments.

This is very much what our lab tried to do with first the matrix creator and then again with the matrix voice, but had to drop entirely due to lack of any official released support and documentation from matrix.io for proper linux audio. If we have to bitbang the SPI bus to get raw ADC samples from the synchronized mic arrays ourselves using some odd sudo os layer in C, and flash the FPGA to ascertain better sampling rates than the course 16K, I'd rather just go with a small usb mic array like this one:

UMA-8 USB mic array

Multichannel USB microphone array for voice command Onboard DSP for beamforming/ noise reduction / echo cancellation / de-reverd / Direction of Arrival (DOA) / Voice Activaty Dectection (VAD) High quality MEMS from Knowles SPH1668LM4H Simple USB control API for Direction of Arrival Flexible Firmware (DSP mode @2ch beamformed, RAW mode @8ch audio from MEMS) Step by step app notes for Google SDK, Alexa-Pi, Cortana, Siri and IBM Watson. Resolution / Sample rate | 24bit @ 11/16/32/44.1/48 kHz Linux Alsa 2.0 compliant

https://www.minidsp.com/products/usb-audio-interface/uma-8-microphone-array

[kdpatino]

Hello @ruffsl and @datascienceteam01

Thank you for your valuable feedback, It helps us to adjust our priorities and keep growing.

Here are answers to some of your concerns:

1- MATRIX Devices as a Linux device: We just released our first version of the MATRIXIO Kernel Modules . Resolution/Sample Rate: 16bit@8 to 96 KHz . https://github.com/matrix-io/matrixio-kernel-modules/releases

If you want to test it, you could run the next python example: https://github.com/matrix-io/matrixio-kernel-modules/blob/master/misc/pyaudio_test.py or use the alsa-utilities (arecord)

2- Kernel Unification: Due to the kernel modules has been released, we made several adjustments in our software, in order to be compatible. Now we are finishing it, and the next week we will release this new version of our software. e.g: https://github.com/matrix-io/matrix-creator-hal/pull/65

3- Support: @datascienceteam01 your welcome to join our community. there you could check some other projects and communicate with us easier. @ruffsl thank you to join our community and post there. I just want to invite you to keep post there any concern, doubt or issue. As I said is easier for us to answer there.

4- Documentation: We know that it is one of the things that we need to improve, you are right. We are making some efforts in order to improve it, but as you report these efforts had not been enough.

Thank You.

Best regards,

Kevin Patino and MATRIX Labs Team.

[alejandrojapkin]

@kdpatino We appreciate you coming in to provide support. My team and I will take a look into the pointers you've given. I would however would like to note that I previously responded to this thread surprised of activity on a one year old open ticket that never took attention; during that time our team has moved to another platform as we had critical deadlines to meet. We felt disappointed and I'd now like to remind you that despite you referring to yourselves as a "community" in those keen "open source" terms, these are boards we paid for, and for which we got zero support when it was critically necessary. This being said, I do personally own one of your boards and I'll give every available tool you commented on a proper testing time.

Thank you for taking the time reviewing this.

[alejandrojapkin]

@kdpatino I've checked the .py you posted, it doesn't seem any different to what we already know. Please check @ruffsl post for an example of a mic array with a solid SDK to illustrate the framework features we were expecting of a board like "Creator". Notice @ruffsl has no affiliation, but we've both come to the same conclusions after experiencing the same shortcomings.

[stoneWeb]

matrix-creator and matrix-voice How to implement DOA function?