search

introlab / odas_web

A desktop visualization GUI for the ODAS library
MIT License
137 stars 53 forks source link

cannot put together odaslive on raspberry and odas_web on a debian #47

Closed DonNils closed 4 years ago

DonNils commented 4 years ago

The ReSpeaker 4-Mic Array is installed on the raspberry according to the guide https://wiki.seeedstudio.com/ReSpeaker_4_Mic_Array_for_Raspberry_Pi All tests running well: python record.py arecord -Dac108 -f S32_LE -r 16000 -c 4 hello.wav I can verify the resulting 4 channel WAV files

Next step is to run odas. The installation on raspberry runs threw without problems.

The odas_web is installed on a debian (buster) pc. The user interface starts is running.

~/GITRepos/ODAS/odas_web$ npm start

odas_studio@0.3.0 start /home/nils/GITRepos/ODAS/odas_web electron .

server listening to {"address":"::","family":"IPv6","port":9000} server listening to {"address":"::","family":"IPv6","port":9001} server listening to {"address":"::","family":"IPv6","port":10000} server listening to {"address":"::","family":"IPv6","port":10010}

On raspberry the configuration file is modified with the IP-number of the odas-web machine. The start of odaslive results in:

./odas/bin/odaslive -vsc ./odas/config/odaslive/respeaker_4_new.cfg +--------------------------------------------+ | ODAS (Open embeddeD Audition System) | +--------------------------------------------+ | Author: Francois Grondin | | Email: francois.grondin2@usherbrooke.ca | | Website: introlab.3it.usherbrooke.ca | | Version: 1.0 | +--------------------------------------------+ | + Initializing configurations...... [Done] | | + Initializing objects............. [Done] | | + Processing....................... Sink pots: Cannot connect to server

Is there a idea how to configure the two systems working together?

Thank you for helping! Nils

GodCed commented 4 years ago

Hi, from what I can see everything seems to be properly configured. Can you ping the PC from the Pi? Are ports 9000, 9001, 10000 and 10010 open?

DonNils commented 4 years ago

Yes I can ping the PC from rarspberry ping 192.168.178.36 PING 192.168.178.36 (192.168.178.36) 56(84) bytes of data. 64 bytes from 192.168.178.36: icmp_seq=1 ttl=64 time=6.43 ms 64 bytes from 192.168.178.36: icmp_seq=2 ttl=64 time=1.40 ms 64 bytes from 192.168.178.36: icmp_seq=3 ttl=64 time=1.41 ms ^C --- 192.168.178.36 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 5ms rtt min/avg/max/mdev = 1.404/3.079/6.428/2.368 ms

or with IPV6: ping 2a02:810b:c500:4a8:4d18:4477:2d99:e417 PING 2a02:810b:c500:4a8:4d18:4477:2d99:e417(2a02:810b:c500:4a8:4d18:4477:2d99:e417) 56 data bytes 64 bytes from 2a02:810b:c500:4a8:4d18:4477:2d99:e417: icmp_seq=1 ttl=254 time=2.43 ms 64 bytes from 2a02:810b:c500:4a8:4d18:4477:2d99:e417: icmp_seq=2 ttl=254 time=1.43 ms 64 bytes from 2a02:810b:c500:4a8:4d18:4477:2d99:e417: icmp_seq=3 ttl=254 time=1.46 ms ^C --- 2a02:810b:c500:4a8:4d18:4477:2d99:e417 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 5ms rtt min/avg/max/mdev = 1.434/1.772/2.426/0.464 ms

and the Ports on PC are open:

sudo lsof -i -P -n | grep LISTEN | grep electron electron 26266 nils 53u IPv6 725452 0t0 TCP :9000 (LISTEN) electron 26266 nils 54u IPv6 725453 0t0 TCP :9001 (LISTEN) electron 26284 nils 69u IPv6 727210 0t0 TCP :10000 (LISTEN) electron 26285 nils 69u IPv6 728152 0t0 TCP :10010 (LISTEN)

GodCed commented 4 years ago

They are open on the PC but is there a chance there's a firewall somewhere in between? I've seen instance of network firewalls blocking traffic. Also, could you provide your ODAS config file?

DonNils commented 4 years ago

both pc and raspberry are connected to my fritzbox. There is no firewall to my knowledge.

here my config-file

Configuration file for ReSpeaker 4 Mic Array (a hat for Raspberry Pi)

version = "2.1";

Raw

raw: {

fS = 16000;
hopSize = 128;
nBits = 32;
nChannels = 4; 

# Input with raw signal from microphones
interface: {
    type = "soundcard";
    card = 1;
    device = 0;
}

}

Mapping

mapping: {

map: (1, 2, 3, 4);

}

General

general: {

epsilon = 1E-20;

size: 
{
    hopSize = 128;
    frameSize = 256;
};

samplerate:
{
    mu = 16000;
    sigma2 = 0.01;
};

speedofsound:
{
    mu = 343.0;
    sigma2 = 25.0;
};

mics = (

    # Microphone 1
    { 
        mu = ( -0.0405, +0.0000, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 2
    { 
        mu = ( +0.0000, +0.0405, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 3
    { 
        mu = ( +0.0405, +0.0000, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 4
    { 
        mu = ( +0.0000, -0.0405, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    }

);

# Spatial filters to include only a range of direction if required
# (may be useful to remove false detections from the floor, or
# limit the space search to a restricted region)
spatialfilters = (

    {

        direction = ( +0.000, +0.000, +1.000 );
        angle = (80.0, 90.0);

    }

);  

nThetas = 181;
gainMin = 0.25;

};

Stationnary noise estimation

sne: {

b = 3;
alphaS = 0.1;
L = 150;
delta = 3.0;
alphaD = 0.1;

}

Sound Source Localization

ssl: {

nPots = 4;
nMatches = 10;
probMin = 0.5;
nRefinedLevels = 1;
interpRate = 4;

# Number of scans: level is the resolution of the sphere
# and delta is the size of the maximum sliding window
# (delta = -1 means the size is automatically computed)
scans = (
    { level = 2; delta = -1; },
    { level = 4; delta = -1; }
);

# Output to export potential sources
potential: {

    # format = "undefined";
    format = "json";

    interface: {
        # type = "blackhole";
        #type = "socket"; ip = "127.0.0.1"; port = 9001;
        #type = "socket"; ip = "192.168.178.5"; port = 9001;
        #type = "socket"; ip = "2a02:810b:c500:4a8:42b5:2be4:c11f:bdf2"; port = 9001;
    type = "socket"; 
ip = "192.168.178.36"; 
port = 9001;
    };
};

};

Sound Source Tracking

sst: { 

# Mode is either "kalman" or "particle"

mode = "kalman";

# Add is either "static" or "dynamic"

add = "dynamic";

# Parameters used by both the Kalman and particle filter

active = (
    { weight = 1.0; mu = 0.3; sigma2 = 0.0025 }
);

inactive = (
    { weight = 1.0; mu = 0.15; sigma2 = 0.0025 }
);

sigmaR2_prob = 0.0025;
sigmaR2_active = 0.0225;
sigmaR2_target = 0.0025;
Pfalse = 0.1;
Pnew = 0.1;
Ptrack = 0.8;

theta_new = 0.9;
N_prob = 5;
theta_prob = 0.8;
N_inactive = ( 150, 200, 250, 250 );
theta_inactive = 0.9;

# Parameters used by the Kalman filter only

kalman: {

    sigmaQ = 0.001;

};

# Parameters used by the particle filter only

particle: {

    nParticles = 1000;
    st_alpha = 2.0;
    st_beta = 0.04;
    st_ratio = 0.5;
    ve_alpha = 0.05;
    ve_beta = 0.2;
    ve_ratio = 0.3;
    ac_alpha = 0.5;
    ac_beta = 0.2;
    ac_ratio = 0.2;
    Nmin = 0.7;

};

target: ();

# Output to export tracked sources
tracked: {

    # format = "undefined";
    format = "json";

    interface: {
        # type = "blackhole";
         type = "socket"; 
ip = "192.168.178.36"; 
port = 9000;
    };

};

}

sss: {

# Mode is either "dds", "dgss" or "dmvdr"

mode_sep = "dds";
mode_pf = "ms";

gain_sep = 1.0;
gain_pf = 10.0;

dds: {

};

dgss: {

    mu = 0.01;
    lambda = 0.5;

};

dmvdr: {

};

ms: {

    alphaPmin = 0.07;
    eta = 0.5;
    alphaZ = 0.8;        
    thetaWin = 0.3;
    alphaWin = 0.3;
    maxAbsenceProb = 0.9;
    Gmin = 0.01;
    winSizeLocal = 3;
    winSizeGlobal = 23;
    winSizeFrame = 256;

};

ss: {

    Gmin = 0.01;
    Gmid = 0.9;
    Gslope = 10.0;

}

separated: {

    fS = 44100;
    hopSize = 512;
    nBits = 16;        
interface: {
    type = "socket";
    ip = "192.168.178.36";
    port = 10000;

}

};

postfiltered: {

    fS = 44100;
    hopSize = 512;
    nBits = 16;        

interface: {
    type = "socket";
    ip = "192.168.178.36";
port = 10010;
      }        

};

}

classify: {

frameSize = 1024;
winSize = 3;
tauMin = 32;
tauMax = 200;
deltaTauMax = 7;
alpha = 0.3;
gamma = 0.05;
phiMin = 0.15;
r0 = 0.2;    

category: {

    format = "undefined";

    interface: {
        type = "blackhole";
    }

}

}

DonNils commented 4 years ago

the netstat from raspberry looks like this:

sudo lsof -i -P -n COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME avahi-dae 379 avahi 12u IPv4 14500 0t0 UDP :5353 avahi-dae 379 avahi 13u IPv6 14501 0t0 UDP :5353 avahi-dae 379 avahi 14u IPv4 14502 0t0 UDP :46692 avahi-dae 379 avahi 15u IPv6 14503 0t0 UDP :56506 dhcpcd 426 root 13u IPv4 15330 0t0 UDP :68 dhcpcd 426 root 16u IPv6 18520 0t0 UDP :546 sshd 498 root 3u IPv4 15419 0t0 TCP :22 (LISTEN) sshd 498 root 4u IPv6 13296 0t0 TCP :22 (LISTEN) sshd 1097 root 3u IPv4 19518 0t0 TCP 192.168.178.5:22->192.168.178.36:33264 (ESTABLISHED) sshd 1106 pi 3u IPv4 19518 0t0 TCP 192.168.178.5:22->192.168.178.36:33264 (ESTABLISHED) sshd 1106 pi 10u IPv6 18814 0t0 TCP [::1]:6010 (LISTEN) sshd 1106 pi 11u IPv4 18815 0t0 TCP 127.0.0.1:6010 (LISTEN)

GodCed commented 4 years ago

Ok everything seems fine networking wise. Is there any output in the terminal on the PC (after the four "listening to...") when you start ODAS?

DonNils commented 4 years ago

No, absolutely silent

DonNils commented 4 years ago

playing around I could "connekt to the ports on PC. I just called in firefox: http://192.168.178.36:10001 and so on for the other ports and the odas_web gives on stdout: <<<<<< ~/GITRepos/ODAS/odas_web$ npm start

odas_studio@0.3.0 start /home/nils/GITRepos/ODAS/odas_web electron .

server listening to {"address":"::","family":"IPv6","port":9000} server listening to {"address":"::","family":"IPv6","port":9001} server listening to {"address":"::","family":"IPv6","port":10000} server listening to {"address":"::","family":"IPv6","port":10010} new client connection from ::ffff:192.168.178.36:42062 connection from ::ffff:192.168.178.36:42062 closed new client connection from ::ffff:192.168.178.36:47042 connection from ::ffff:192.168.178.36:47042 closed new client connection from ::ffff:192.168.178.36:60886 connection from ::ffff:192.168.178.36:60886 closed new client connection from ::ffff:192.168.178.36:60888 connection from ::ffff:192.168.178.36:60888 closed new client connection from ::ffff:192.168.178.36:40554 connection from ::ffff:192.168.178.36:40554 closed

<<<<<<<<<<

It seems to me this firefox test fails using a different computer, raspberry or tablet ...

Is it possible that odas_web only connect to localhost connektions?

GodCed commented 4 years ago

At least it is not supposed to. I mainly used it with remote machines. At this point I would take a look at your network, this really looks like a firewall problem because there is not even a connection attempt between ODAS and ODAS Studio.

You may try curl on the Pi to reproduce the Firefox test:

$ curl 192.168.178.36:9000
DonNils commented 4 years ago

OK, I´ve got it!

Indeed my debian PC uses an default iptable configuration... alias firewall I opened it with ufw and odaslive and odas_web are talking together

sudo ufw allow 9000 sudo ufw allow 9001 sudo ufw allow 10010 sudo ufw allow 10000

I am sure, I could set it more safely.

Thank you so much Nils

GodCed commented 4 years ago

My pleasure, glad to hear it's working properly now!