ODAS not outputting measurements for ReSpeaker Mic Array v2.0

[ameliefroessl]

Hi! I'm trying to setup the ReSpeaker Mic Array v2.0 and get Sound Source localization data, however when I start and launch ODAS, there is no data being shown. I am using the respeaker_4_mic_array.cfg file. I changed the 'SSS' part to reflect the config file changes here (https://github.com/introlab/odas_web#configure-odas). and I'm using the /odas/bin/odaslive as a core. This is the output I get:

odas_studio@0.3.0 start /home/amelie/odas_web electron .

server listening to {"address":"::","family":"IPv6","port":9000} server listening to {"address":"::","family":"IPv6","port":9001} server listening to {"address":"::","family":"IPv6","port":10000} server listening to {"address":"::","family":"IPv6","port":10010} Gtk-Message: 11:59:12.165: GtkDialog mapped without a transient parent. This is discouraged. Gtk-Message: 11:59:57.253: GtkDialog mapped without a transient parent. This is discouraged. received launch command /home/amelie/odas/bin/odaslive /home/amelie/odas/config/odaslive/respeaker_4_mic_array_changed.cfg new client connection from ::ffff:127.0.0.1:60208 new client connection from ::ffff:127.0.0.1:44968 new client connection from ::ffff:127.0.0.1:45572 new client connection from ::ffff:127.0.0.1:36952 connection from ::ffff:127.0.0.1:45572 closed connection from ::ffff:127.0.0.1:36952 closed connection from ::ffff:127.0.0.1:60208 closed connection from ::ffff:127.0.0.1:44968 closed

Thanks in advance!

[GodCed]

Hi, it seems like ODAS is starting then stopping immediately (new connection, then connection closed).

Could you launch ODAS Live (the GUI) and ODAS (the processing core) in separated terminals then provide both outputs? Instead of launching ODAS trough the GUI "launch" button, run:

/path/to/odaslive -c /path/to/config.cfg

to start ODAS Core.

[ameliefroessl]

Ok I tried starting the ODAS Live with npm start and then when I try to start ODAS Core with:

./odaslive -c /odas/config/odaslive/respeaker_4_mic_array.cfg

I get:

Source hops: Cannot set sample format: Invalid argument

and the output for ODAS Live is still:

odas_studio@0.3.0 start /home/amelie/odas_web electron .

server listening to {"address":"::","family":"IPv6","port":9000} server listening to {"address":"::","family":"IPv6","port":9001} server listening to {"address":"::","family":"IPv6","port":10000} server listening to {"address":"::","family":"IPv6","port":10010} new client connection from ::ffff:127.0.0.1:45950 new client connection from ::ffff:127.0.0.1:43102 new client connection from ::ffff:127.0.0.1:33246 new client connection from ::ffff:127.0.0.1:55024 connection from ::ffff:127.0.0.1:43102 closed connection from ::ffff:127.0.0.1:45950 closed connection from ::ffff:127.0.0.1:33246 closed connection from ::ffff:127.0.0.1:55024 closed new client connection from ::ffff:127.0.0.1:45958 new client connection from ::ffff:127.0.0.1:43110 new client connection from ::ffff:127.0.0.1:33254 new client connection from ::ffff:127.0.0.1:55032 connection from ::ffff:127.0.0.1:43110 closed connection from ::ffff:127.0.0.1:45958 closed connection from ::ffff:127.0.0.1:33254 closed connection from ::ffff:127.0.0.1:55032 closed new client connection from ::ffff:127.0.0.1:43114 new client connection from ::ffff:127.0.0.1:45962 new client connection from ::ffff:127.0.0.1:33266 new client connection from ::ffff:127.0.0.1:55044 connection from ::ffff:127.0.0.1:43114 closed connection from ::ffff:127.0.0.1:33266 closed connection from ::ffff:127.0.0.1:45962 closed connection from ::ffff:127.0.0.1:55044 closed

Ill attach the config file for reference, I haven't changed anything expect the IPs.

respeaker_4_mic_array_changed.txt

[GodCed]

Hi, you started your configuration from the wrong file. You need to use respeaker.cfg (for respeaker round array) and not respeaker_4_mic_array.cfg (for raspberry PI hat). The cards use different raw audio format and the microphones locations are not the same.

[ameliefroessl]

Ah ok, now i get :

Source hops: Cannot set channel count: Invalid argument

I tried changing from 8 to 6 (I think the ReSpeaker has 6 channel output?).. but then it gives me a segmentation fault.

[GodCed]

Did you perform the mic array firmware as stated in the Seeed Studio wiki?

[ameliefroessl]

yes i did that ..

[GodCed]

Did you use the config file from the tutorial?

[ameliefroessl]

Ok that works! Cool thanks!

[ameliefroessl]

Is there any way to get the azimuth/elevation in some other form than through the GUI?

[GodCed]

Yes, all data to the GUI is passed trough sockets. The GUI is the server and ODAS is the client. Azimuth and elevation are encoded in JSON. You can write your own application implementing a socket server to receive the data or output the JSON to a text file by changing the sink type in the ODAS config file.

[ameliefroessl]

Ok. What part of the config file is that?

[GodCed]

It's the part where you changed the IPs. Azimut and elevation is under the SST module. You must change the type and then add the proper parameters.

[ameliefroessl]

ah ok! Cool thanks!

[ameliefroessl]

Sorry one more question. Do you know how accurate the sound source location estimates are for ODAS with the ReSpeaker microphone?

[GodCed]

The accuracy greatly depends of the parameters used. Although it is on a different microphone array, section 6.1.1 of the ODAS paper is of interest here. It gives a good overview of the relation between sound source tracking parameters and tracking accuracy.

[GodCed]

From what I understand you're trying to connect a socket client to the ODAS Studio (GUI) socket server. It's expected that you receive nothing because the ODAS Studio server only receives data.

You need to implement a socket server and then start ODAS so that it connects to your server instead of ODAS Studio.

[ameliefroessl]

Hi, sorry I deleted the question because I figured it out and now have a socket server to receive the data from ODAS the client. I also wasn't aware that I need to be listening to all 4 of the ports (9000,9001, 10000, 10010 I believe they are). Otherwise ODAS will quit. But thanks for the answer!

[GodCed]

Actually you don’t have to listen to the four ports. You can change the sink configurations of the modules you don’t need to a « blackhole » type and it won’t try to connect. Search for the ports number in the configuration file to easily find them.

[ameliefroessl]

Ah ok. That makes sense. Is there a way actually to get the raw audio data in a similar way ? Because I can't connect to the speaker through ODAS and Ros at the same time. I think the device can only be accessed by one application at a time.

[GodCed]

Hi, unfortunately ODAS does not support sinking the raw audio. You are right that the sound card can only be opened by one application at the time.

[ameliefroessl]

Ok thanks for the information!

[ameliefroessl]

Is there a way to change the sensitivity of what is considered a sound source ? I know there are some parameters in the config file, although in your wiki you say to leave them at the values they are.

[GodCed]

Yes there is. In the tracking module configuration (SST) look for the active and inactive entries. The mu value is the energy level that is considered as an active (begin tracking) or inactive (end tracking) sound source.

You can gage the appropriate settings by looking at the energy level of the potential sources (SSL output or blue->red gradient square in the GUI). As an exemple, let's say you are picking up fan noise from a computer with an energy level of around 0.4 and a voice at an energy level of 0.7, you'll want active mu to 0.7 and inactive mu to 0.4 as a starting point for tuning.

You can also play with the corresponding weights so that the algorithm "prefers" to consider a source as active or inactive. Lastly, what I describe as a source is an aggregate of energy that shows up as a spot with a lot of reddish energy square in the GUI.

I hope it makes sense.

[ameliefroessl]

Yes it does! Cool thanks! :)

[ameliefroessl]

I hope you don't mind me asking all these questions in the same thread.... I was also wondering what the exact coordinate system orientation is given the Respeaker hardware. From what I gather, the coordinate system is aligned with the physical microphones' position. And the microphone to the left of the usb port is positive x, the one to the right is positive y, positive z would be straight up. Is this correct?

[GodCed]

Hi, no problem for the questions.

As for the coordinate system, it matches the physical microphones measurements as you guessed. As is, ODAS measurements are produced in the same coordinates system used by the user while defining the microphones positions. Sadly, I didn't write the configuration file (in fact I don't have the board) so I cannot confirm your conclusions.

[ameliefroessl]

Ah ok. Do you know who wrote the config file you sent me by any chance?

[GodCed]

That would be respeaker themselves. But really if the microphones measurements match then you know you have the proper coordinate system.

[deeprobo-dev]

sudo aplay -l run this in terminal to know what is your card no and put it in odas.cfg file it will mostly solve the issue.

[Shirinkori]

Did you use the config file from the tutorial?

I was working on the DOA estimation using 4-mic linear ReSpeaker array, can you tell what changes I need to do in the configuration?