MauroPfister commented 5 years ago

Hi,

I wrote a little python script with a socket server to receive the SST data streamed by odaslive via a socket client. During some tests I noticed that with a buffer of 2048 bytes (actually down to 1,567) I receive a string containing 4 times timestamp + src data as shown below. When I lower the buffer size I only receive junks of the SST data but never a complete json with timestamp + src data.

Hence I was wondering if that behaviour (4 times timestamp + src data) is intended or if I made a mistake in my program? And if so, why do you send the data grouped together instead of just sending one timestamp + src data at a time?

Thanks a lot for your help.

{
    "timeStamp": 174,
    "src": [
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 }
    ]
}
{
    "timeStamp": 175,
    "src": [
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 }
    ]
}
{
    "timeStamp": 176,
    "src": [
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 }
    ]
}
{
    "timeStamp": 177,
    "src": [
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000 }
    ]
}

FrancoisGrondin commented 5 years ago

Typically when you send multiple messages back to back to a socket, you can't guarantee they will be received individually when you read the socket.

baynaa7 commented 5 years ago

Hello, Im using input.raw (mixture signals are first recorded in audacity and converted to raw with sox and recorded .wav file has float32 and 8 channels with fs=16k) and with respeaker.cfg in odas. However, when I run odaslive then tracks.txt file was generated. However, all x,y and z values are zero as shown above sample. Does this mean odas did not detected any source directions? or any other problem? Also post_filtered.raw and separated.raw have 0 values. here is my respeaker.cfg file: `# Configuration file for ReSpeaker circular sound card

version = "2.1";

Raw

raw: {

fS = 16000;
hopSize = 128;
nBits = 16;
nChannels = 8; 

# Input with raw signal from file
interface: {
    type = "file";
    path = "input.raw";
};

}

Mapping

mapping: {

map: (1, 2, 3, 4, 5, 6, 7);

}

General

general: {

epsilon = 1E-20;

size: 
{
    hopSize = 128;
    frameSize = 256;
};

samplerate:
{
    mu = 16000;
    sigma2 = 0.01;
};

speedofsound:
{
    mu = 343.0;
    sigma2 = 25.0;
};

mics = (

    # Microphone 1
    { 
        mu = ( +0.0000, +0.0000, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 2
    { 
        mu = ( -0.0160, +0.0277, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 3
    { 
        mu = ( -0.0320, +0.0000, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 4
    { 
        mu = ( -0.0160, -0.0277, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 5
    { 
        mu = ( +0.0160, -0.0277, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );        
    },

    # Microphone 6
    { 
        mu = ( +0.0320, +0.0000, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );        
    },

    # Microphone 7
    { 
        mu = ( +0.0160, +0.0277, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    }

);

# Spatial filters to include only a range of direction if required
# (may be useful to remove false detections from the floor, or
# limit the space search to a restricted region)
spatialfilters = (

    {

        direction = ( +0.000, +0.000, +1.000 );
        angle = (80.0, 90.0);

    }

);  

nThetas = 181;
gainMin = 0.25;

};

Stationnary noise estimation

sne: {

b = 3;
alphaS = 0.1;
L = 150;
delta = 3.0;
alphaD = 0.1;

}

Sound Source Localization

ssl: {

nPots = 4;
nMatches = 10;
probMin = 0.5;
nRefinedLevels = 1;
interpRate = 4;

# Number of scans: level is the resolution of the sphere
# and delta is the size of the maximum sliding window
# (delta = -1 means the size is automatically computed)
scans = (
    { level = 2; delta = -1; },
    { level = 4; delta = -1; }
);

# Output to export potential sources
potential: {
    format = "undefined";
    interface: {
        type = "blackhole"; 
    };
};

};

Sound Source Tracking

sst: {

# Mode is either "kalman" or "particle"

mode = "particle";

# Add is either "static" or "dynamic"

add = "dynamic";

# Parameters used by both the Kalman and particle filter

active = (
    { weight = 1.0; mu = 0.3; sigma2 = 0.0025 }
);

inactive = (
    { weight = 1.0; mu = 0.15; sigma2 = 0.0025 }
);

sigmaR2_prob = 0.0025;
sigmaR2_active = 0.0225;
sigmaR2_target = 0.0025;
Pfalse = 0.1;
Pnew = 0.1;
Ptrack = 0.8;

theta_new = 0.9;
N_prob = 5;
theta_prob = 0.8;
N_inactive = ( 150, 200, 250, 250 );
theta_inactive = 0.9;

# Parameters used by the Kalman filter only

kalman: {

    sigmaQ = 0.001;

};

# Parameters used by the particle filter only

particle: {

    nParticles = 1000;
    st_alpha = 2.0;
    st_beta = 0.04;
    st_ratio = 0.5;
    ve_alpha = 0.05;
    ve_beta = 0.2;
    ve_ratio = 0.3;
    ac_alpha = 0.5;
    ac_beta = 0.2;
    ac_ratio = 0.2;
    Nmin = 0.7;

};

target: ();

# Output to export tracked sources

tracked: {

    format = "json";

    interface: {
        type = "file";
        path = "tracks.txt";
    };

};

}

sss: {

# Mode is either "dds", "dgss" or "dmvdr"

mode_sep = "dds";
mode_pf = "ms";

gain_sep = 1.0;
gain_pf = 10.0;

dds: {

};

dgss: {

    mu = 0.01;
    lambda = 0.5;

};

dmvdr: {

};

ms: {

    alphaPmin = 0.07;
    eta = 0.5;
    alphaZ = 0.8;        
    thetaWin = 0.3;
    alphaWin = 0.3;
    maxAbsenceProb = 0.9;
    Gmin = 0.01;
    winSizeLocal = 3;
    winSizeGlobal = 23;
    winSizeFrame = 256;

};

ss: {

    Gmin = 0.01;
    Gmid = 0.9;
    Gslope = 10.0;

}

 separated: {

    fS = 16000;
    hopSize = 128;
    nBits = 16;        

    interface: {
        type = "file";
        path = "separated.raw";
    };        

};

postfiltered: {

    fS = 16000;
    hopSize = 128;
    nBits = 16;        
    gain = 10.0;

    interface: {
        type = "file";
        path = "postfiltered.raw";
    };        

};

}

classify: {

frameSize = 1024;
winSize = 3;
tauMin = 32;
tauMax = 200;
deltaTauMax = 7;
alpha = 0.3;
gamma = 0.05;
phiMin = 0.15;
r0 = 0.2;    

category: {

    format = "undefined";

    interface: {
        type = "blackhole";
    }

}

} ` Can I see the result of input.raw in odas_web? because I cannot see anything on odas_web size. Thanks,

Junye97 commented 5 years ago

@pcub I'm not sure about the sst output since I am also trying to figure it out, but I can spot some other issues with your config file:

Most of your output interface are "blackhole" so I reckon you wouldn't see anything at all.
For the input file path, I would suggest you put the full file path, e.g. /home/user/odas/bin/input.raw, and the same for the outputs. If you don't put the full path odas_web cannot read the input (and it does not spit out errors for this).

baynaa7 commented 5 years ago

Thanks Junye97, I will test that.

Junye97 commented 5 years ago

@pcub please share your findings with the SST output.... I am still investigating what went wrong with my config. Thank you

MauroPfister commented 5 years ago

Typically when you send multiple messages back to back to a socket, you can't guarantee they will be received individually when you read the socket.

Yes, I figured that out while reading up on TCP sockets. I ended up rewriting the javascript code of odas_web in python to parse the incoming json data. The parsing function can be found at https://github.com/introlab/odas_web/blob/66fda0dae550e87afe065e2eeb21152ea6ea1ddc/servers.js#L31-L71. I thought this might be useful to others.

@pcub: I don't want to be rude, but your issue does not seem related to my question. Maybe it's best if you open a separate issue for it.

marcus905 commented 4 years ago

Sorry for the off-topic post, but I'm having some of the same issues and I can't seem to find a good source of documentation on how odas works, except reimplementing odas_web.

@MauroPfister by any chance is your python code that decodes the odas output available to read or open source? If so can I have it?

Thank you!

Apolotary commented 4 years ago

@marcus905 did you manage to get any info? Just ran into the same problem myself, would like to use ODAS in my project, but can't figure out how the info is being sent and in what format

marcus905 commented 4 years ago

Not yet, I'm still working on it, but I'll let you know

ameliefroessl commented 4 years ago

@MauroPfister I would also be interested in seeing how you receive the data in a python script. I am able to connect to the port which is outputting the SST data, however it seems like no data is being streamed. However, the data is definitely being received by odas_web.

Funchhy commented 3 years ago

For me, if i specify IP and Port(in this case 5027), as well as configure it to be TCP + json in the config file i can get Data via:

import socket
import sys
import time

socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.bind(('', 5027))
socket.listen(5)
client, address = socket.accept()
print("{} connected".format( address ))
while True:
    response = client.recv(512)   
    print(response)
    time.sleep(1)

print ("close")
client.close()
stock.close()

Femtofirst commented 1 year ago

Hi,