lembit1 commented 3 years ago

Hello, I would like to export ssl and sst output to files as well sending to the socket for ODAS Studio, based on modifying respeaker_usb_4_mic_array.cfg . To start, I exported tracked sources interface: {

HACK 2b

        **type = "file"; path = "tracks.txt";
        #type = "socket"; ip = "127.0.0.1"; port = 9000;**
        # type = "terminal";

and ODAS Studio runs (nothing responding on graphics as I expected), tracks.txt is created but it is empty. I note that minidsp.cfg sends ssl to "blackhole" and sst to "terminal" Q1: is it possible for ODAS Studio to export to file and socket simultaneously? How? Q2: does the separated.raw output have a gain parameter? Q3: what program consumes minidsp.cfg? Q4: what's cooking in the "classify" tail of the .cfg? Thank you!

lembit1 commented 3 years ago

Update: I fixed physical mic connection and now tracks.txt is populated. Here's typical json message: { "timeStamp": 1202, "src": [ { "id": 7, "tag": "dynamic", "x": -0.004, "y": -0.858, "z": 0.513, "activity": 0.000 }, { "id": 22, "tag": "dynamic", "x": 0.825, "y": -0.006, "z": 0.565, "activity": 0.999 }, { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 }, { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 } ] } Q5: what is the timeStamp unit? If it is an async count, can you guide me how to replace with system Unix time? Q6: is the activity value related to some movement-related parameter? Can you describe? Q7: are there any other formats available other than json? when I divert the SSL export to a file, the json messages look like { "timeStamp": 1146, "src": [ { "x": 0.552, "y": -0.648, "z": 0.525, "E": 0.442 }, { "x": 0.672, "y": -0.719, "z": 0.175, "E": 0.359 }, { "x": 0.460, "y": -0.840, "z": 0.288, "E": 0.126 }, { "x": 0.728, "y": -0.453, "z": 0.514, "E": 0.058 } ] } Q8: can you parse this please? Thanks much!

GodCed commented 3 years ago

Q1: you can have one sink interface per output. So you can't send the same data to both a file and ODAS Studio.

Q2: Yes. It is gain_sep under the SSS module. You also have gain_pf for post-filtered separated audio.

Q3: odaslive, which does the actual processing. ODAS studio is simply a GUI which receive data from the odaslive process trough sockets.

Q4: no clue on that one. It was non existant while I coded ODAS Studio as an IntRoLab's intern. You'll need to ask on the ODAS repo.

Q5: it's an incremental message counter. You must edit the odas library source code to change it. I would advise you add a secondary field instead of modifying the existing timestamp field, I'm pretty sure it would reduce the chance of messing up odas pocessing pipeline.

Q6: it goes from 0 to 1 and indicate "how much the tracking filter consider the source as producing sound". when it is first heard it starts at one, then when the sound stops it "slowly" goes back to zero. The idea is allow the source to survive if the sound drops momentarily to allow tracking of intermittent source such as a person talking.

A7: only JSON for tracking data.

Q8: timeStamp, incremental as discussed; src, array of directions (x,y,z unit vector) from which sound energy (E) is coming. If E is high enough, long enough, this direction becomes a tracked source and it appears in tracking data (SST output).

For reference, here is the link to the ODAS repo https://github.com/introlab/odas.

introlab / odas_web

socket AND file output #65

HACK 2b