Problem running in windows 10 and linux

[gary0351]

I installed nodejs/npm/electron. Got past gyp issue with install.

Windows distribution exe will not bring up web page. It lists ports to monitor on startup and does nothing else.

I set up a web app version of ODAS web using 'npm start' which brings up web page, displays noise and speech sources but hangs on recording. Switched to latest release which does same thing.

Perhaps I missed something when installing nodejs, etc. I reinstalled to make sure.

I set up same code running with ubuntu on virtualBox. It runs and sort of records audio <1 sec. Which is better OS - win10 or linux or macOS? What was web server developed on? Any insight would be appreciated. Attached log files from win execution. Thanks Gary ms-cmd-prompt.log 2018-11-01T15_06_40_420Z-debug.log

[GodCed]

Seems like a lot of things are not running smoothly there. I am not a node expert in any way so I can't really pinpoint the problem from your logs.

Pre-compiled release are outdated, so that's that.

Web server was developed on both Mac and Linux. I also ran it on Windows. I don't think there is a "recommended os". Virtual Machines may be underpowered and/or experience problems with WebGL drivers, so I would try to avoid them. I think your problem on Ubuntu is simply that the recorder process can't keep up because system ressources are hoarded by the UI updates (the charts).

My advice would be to try to use Node 8 instead of 10, as it is the Node version the app was developed on and some package seems to not upgrade very well. (Mainly the gyp things)

[gary0351]

Cedric,

Appreciate the quick response. I’ll try running with Node8 first.

I asked about the OS thinking that it was developed on a *nix platform. If windows doesn’t work, I’ll set up a dedicated Linux machine or find a mac.

Thanks again.

Gary

From: Cédric Godin Sent: Thursday, November 1, 2018 7:12 PM To: introlab/odas_web Cc: gary0351; Author Subject: Re: [introlab/odas_web] Problem running in windows 10 and linux (#22)

Seems like a lot of things are not running smoothly there. I am not a node expert in any way so I can't really pinpoint the problem from your logs. Pre-compiled release are outdated, so that's that. Web server was developed on both Mac and Linux. I also ran it on Windows. I don't think there is a "recommended os". Virtual Machines may be underpowered and/or experience problems with WebGL drivers, so I would try to avoid them. I think your problem on Ubuntu is simply that the recorder process can't keep up because system ressources are hoarded by the UI updates (the charts). My advice would be to try to use Node 8 instead of 10, as it is the Node version the app was developed on and some package seems to not upgrade very well. (Mainly the gyp things) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[gary0351]

Cedric,

Have some questions about ODAS.

Post Filtered and Separated Sources streams have 4 channels interleaved so 8 bytes includes one 16 bit sample in order for each channel. Data stream is Network Byte Ordered (Big Endian). Is that correct?

I adapted the respeaker .cfg file for ODAS running on a respeaker core v2 demo board and it feeds audio to ODAS Web just fine.

I am feeding PF (or SS) stream to android app using TCP socket and getting poor audio – does not sound like speech at all. Track and other streams currently go to ODAS Web. The app has a receive buffer size of 2048 bytes. I see data input of up to 1448 bytes from ODAS. Channel 0 is extracted from receive buffer to feed audio to another service. I’ll process the other channels when I get this working.

The respeaker and android processors should be adequate for what I want to do.

Do you have suggestions/recommendations to improve audio quality. Are there buffer size changes or other settings in .cfg file that would help? Should the hopsize be set to something else? Current cfg parameters for PF and SS used with ODAS Web and android app are:

separated: {

    fS = 16000;
    hopSize = 512;
    nBits = 16;

    interface: {
            type = "socket";
            ip = "192.168.1.10";
            port = 10000;
    }

};

postfiltered: {

    fS = 16000;
    hopSize = 512;
    nBits = 16;

    interface: {
            type = "socket";
            ip = "192.168.1.14";
            port = 10010;
    }

};

I really appreciate any suggestions you might offer.

Thanks, Gary

From: Cédric Godin Sent: Thursday, November 1, 2018 7:12 PM To: introlab/odas_web Cc: gary0351; Author Subject: Re: [introlab/odas_web] Problem running in windows 10 and linux (#22)

Seems like a lot of things are not running smoothly there. I am not a node expert in any way so I can't really pinpoint the problem from your logs. Pre-compiled release are outdated, so that's that. Web server was developed on both Mac and Linux. I also ran it on Windows. I don't think there is a "recommended os". Virtual Machines may be underpowered and/or experience problems with WebGL drivers, so I would try to avoid them. I think your problem on Ubuntu is simply that the recorder process can't keep up because system ressources are hoarded by the UI updates (the charts). My advice would be to try to use Node 8 instead of 10, as it is the Node version the app was developed on and some package seems to not upgrade very well. (Mainly the gyp things) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[GodCed]

Post Filtered and Separated Sources streams have 4 channels interleaved so 8 bytes includes one 16 bit sample in order for each channel.

Yes, that's right.

Data stream is Network Byte Ordered (Big Endian).

I'm like 90% sure that is correct. You may want to check the ODAS repo if testing both is time consuming.

I adapted the respeaker .cfg file for ODAS running on a respeaker core v2 demo board and it feeds audio to ODAS Web just fine.

Good, so we know your config file is properly written.

I am feeding PF (or SS) stream to android app using TCP socket and getting poor audio – does not sound like speech at all. Track and other streams currently go to ODAS Web. The app has a receive buffer size of 2048 bytes. I see data input of up to 1448 bytes from ODAS. Channel 0 is extracted from receive buffer to feed audio to another service. I’ll process the other channels when I get this working.

Two things I would check. Sample format that your Android APP accepts, wether it's signed, unsigned or float. Timing with the sample, packets always arrive in order in socket, but they can be "glued" together. As packet 1, 2 then 3 can arrive as 1+2 then 3. Thus, it is important to never discard bytes and to start from the first byte sent by ODAS while parsing. You may want to take a look at recordings.js and audio-recorder.js for inspiration.

[gary0351]

I will follow up on your suggestions.

Thanks again, Gary

From: Cédric Godin Sent: Monday, December 3, 2018 6:02 PM To: introlab/odas_web Cc: gary0351; Author Subject: Re: [introlab/odas_web] Problem running in windows 10 and linux (#22)

Post Filtered and Separated Sources streams have 4 channels interleaved so 8 bytes includes one 16 bit sample in order for each channel. Yes, that's right. Data stream is Network Byte Ordered (Big Endian). I'm like 90% sure that is correct. You may want to check the ODAS repo if testing both is time consuming. I adapted the respeaker .cfg file for ODAS running on a respeaker core v2 demo board and it feeds audio to ODAS Web just fine. Good, so we know your config file is properly written. I am feeding PF (or SS) stream to android app using TCP socket and getting poor audio – does not sound like speech at all. Track and other streams currently go to ODAS Web. The app has a receive buffer size of 2048 bytes. I see data input of up to 1448 bytes from ODAS. Channel 0 is extracted from receive buffer to feed audio to another service. I’ll process the other channels when I get this working. Two things I would check. Sample format that your Android APP accepts, wether it's signed, unsigned or float. Timing with the sample, packets always arrive in order in socket, but they can be "glued" together. As packet 1, 2 then 3 can arrive as 1+2 then 3. Thus, it is important to never discard bytes and to start from the first byte sent by ODAS while parsing. You may want to take a look at recordings.js and audio-recorder.js for inspiration. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[gary0351]

For some reason the ODAS software stops transmitting audio after about 20 seconds. The audio I do get is very clear with low latency. Is there a way to debug what is happening internally?

Attached copy of the config file for reference. Is there some setting that could be terminating the audio transmission?

These are the instructions (https://respeaker.io/6_mic_array/) I followed to build the software for the respeaker board running Debian: sudo apt-get install libfftw3-dev libconfig-dev libasound2-dev sudo apt-get install cmake git clone https://github.com/introlab/odas.git mkdir odas/build cd odas/build cmake .. make

Recommendations?

Thanks, Gary

From: Cédric Godin Sent: Monday, December 3, 2018 6:02 PM To: introlab/odas_web Cc: gary0351; Author Subject: Re: [introlab/odas_web] Problem running in windows 10 and linux (#22)

Post Filtered and Separated Sources streams have 4 channels interleaved so 8 bytes includes one 16 bit sample in order for each channel. Yes, that's right. Data stream is Network Byte Ordered (Big Endian). I'm like 90% sure that is correct. You may want to check the ODAS repo if testing both is time consuming. I adapted the respeaker .cfg file for ODAS running on a respeaker core v2 demo board and it feeds audio to ODAS Web just fine. Good, so we know your config file is properly written. I am feeding PF (or SS) stream to android app using TCP socket and getting poor audio – does not sound like speech at all. Track and other streams currently go to ODAS Web. The app has a receive buffer size of 2048 bytes. I see data input of up to 1448 bytes from ODAS. Channel 0 is extracted from receive buffer to feed audio to another service. I’ll process the other channels when I get this working. Two things I would check. Sample format that your Android APP accepts, wether it's signed, unsigned or float. Timing with the sample, packets always arrive in order in socket, but they can be "glued" together. As packet 1, 2 then 3 can arrive as 1+2 then 3. Thus, it is important to never discard bytes and to start from the first byte sent by ODAS while parsing. You may want to take a look at recordings.js and audio-recorder.js for inspiration. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

Configuration file for ReSpeaker circular sound card

version = "2.1";

Raw

raw: { fS = 16000; hopSize = 128; nBits = 16; nChannels = 8;

Input with raw signal from microphones

interface: {
    type = "soundcard";
    card = 0;
    device = 0;
}

}

Mapping

mapping: {

map: (1, 2, 3, 4, 5, 6, 7);

map: (1, 2, 3, 4, 5, 6);

}

General

general: { epsilon = 1E-20; size: { hopSize = 128; frameSize = 256; }; samplerate: { mu = 16000; sigma2 = 0.01; }; speedofsound: { mu = 343.0; sigma2 = 25.0; }; mics = (

Microphone 1

    {
        mu = ( +0.0000, +0.0000, +0.0000 );
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },
    # Microphone 2
    {
        mu = ( -0.0160, +0.0277, +0.0000 );
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },
    # Microphone 3
    {
        mu = ( -0.0320, +0.0000, +0.0000 );
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },
    # Microphone 4
    {
        mu = ( -0.0160, -0.0277, +0.0000 );
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },
    # Microphone 5
    {
        mu = ( +0.0160, -0.0277, +0.0000 );
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },
    # Microphone 6
    {
        mu = ( +0.0320, +0.0000, +0.0000 );
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    #},
    }
    # Microphone 7
    # {
    #     mu = ( +0.0160, +0.0277, +0.0000 );
    #     sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
    #     direction = ( +0.000, +0.000, +1.000 );
    #     angle = ( 80.0, 90.0 );
    # }
);
# Spatial filters to include only a range of direction if required
# (may be useful to remove false detections from the floor, or
# limit the space search to a restricted region)
spatialfilters = (
    {
        direction = ( +0.000, +0.000, +1.000 );
        angle = (80.0, 90.0);

    }
);
nThetas = 181;
gainMin = 0.25;

};

Stationnary noise estimation

sne: { b = 3; alphaS = 0.1; L = 150; delta = 3.0; alphaD = 0.1; }

Sound Source Localization

ssl: { nPots = 4; nMatches = 10; probMin = 0.5; nRefinedLevels = 1; interpRate = 4;

Number of scans: level is the resolution of the sphere

# and delta is the size of the maximum sliding window
# (delta = -1 means the size is automatically computed)
scans = (
    { level = 2; delta = -1; },
    { level = 4; delta = -1; }
);
# Output to export potential sources
potential: {
      # format = "json";
      format = "undefined";
      interface: {
            type = "blackhole"
            # type = "socket";
            # ip = "192.168.1.10";
            # port = 9001;
            };
};

};

Sound Source Tracking

sst: {

Mode is either "kalman" or "particle"

mode = "kalman";
# Add is either "static" or "dynamic"
add = "dynamic";
# Parameters used by both the Kalman and particle filter
active = (
    { weight = 1.0; mu = 0.3; sigma2 = 0.0025 }
);
inactive = (
    { weight = 1.0; mu = 0.15; sigma2 = 0.0025 }
);
sigmaR2_prob = 0.0025;
sigmaR2_active = 0.0225;
sigmaR2_target = 0.0025;
Pfalse = 0.1;
Pnew = 0.1;
Ptrack = 0.8;
theta_new = 0.9;
N_prob = 5;
theta_prob = 0.8;
N_inactive = ( 150, 200, 250, 250 );
theta_inactive = 0.9;
# Parameters used by the Kalman filter only
kalman: {
    sigmaQ = 0.001;
};
# Parameters used by the particle filter only
particle: {
    nParticles = 1000;
    st_alpha = 2.0;
    st_beta = 0.04;
    st_ratio = 0.5;
    ve_alpha = 0.05;
    ve_beta = 0.2;
    ve_ratio = 0.3;
    ac_alpha = 0.5;
    ac_beta = 0.2;
    ac_ratio = 0.2;
    Nmin = 0.7;
};
target: ();
# Output to export tracked sources
tracked: {
      format = "json";
      interface: {
            type = "socket";
            ip = "192.168.1.14";
            port = 9000;
    };
};

} sss: {

Mode is either "dds", "dgss" or "dmvdr"

mode_sep = "dds";
mode_pf = "ms";
gain_sep = 1.0;
gain_pf = 10.0;
dds: {
};
dgss: {
    mu = 0.01;
    lambda = 0.5;
};
dmvdr: {
};
ms: {
    alphaPmin = 0.07;
    eta = 0.5;
    alphaZ = 0.8;
    thetaWin = 0.3;
    alphaWin = 0.3;
    maxAbsenceProb = 0.9;
    Gmin = 0.01;
    winSizeLocal = 3;
    winSizeGlobal = 23;
    winSizeFrame = 256;
};
ss: {
    Gmin = 0.01;
    Gmid = 0.9;
    Gslope = 10.0;
}
separated: {
    fS = 16000;
    hopSize = 512;
    nBits = 16;
    interface: {
            type = "socket";
            ip = "192.168.1.14";
            port = 10000;
    }
};
postfiltered: {
    fS = 16000;
    hopSize = 512;
    nBits = 16;
    interface: {
            type = "socket";
            ip = "192.168.1.10";
            port = 10010;
    }
};

} classify: { frameSize = 1024; winSize = 3; tauMin = 32; tauMax = 200; deltaTauMax = 7; alpha = 0.3; gamma = 0.05; phiMin = 0.15; r0 = 0.2; category: { format = "undefined"; interface: { type = "blackhole"; } } }

[GodCed]

From what I seen in the past, this usually happens when either ODAS cannot process the audio in real time or the receiving end cannot keep up with the incoming data.

ODAS internal or socket buffer fill up and processing stops. I would take a look at CPU usage on the machine that’s running ODAS and on the machine receiving audio to see if any of them is saturated.

[gary0351]

Is there a way to turn off postfiltered audio and potential sources? I only need tracked and separated sources.

From: Cédric Godin Sent: Monday, January 7, 2019 10:15 PM To: introlab/odas_web Cc: gary0351; Author Subject: Re: [introlab/odas_web] Problem running in windows 10 and linux (#22)

From what I seen in the past, this usually happens when either ODAS cannot process the audio in real time or the receiving end cannot keep up with the incoming data. ODAS internal or socket buffer fill up and processing stops. I’ll take a look at CPU usage on the machine that’s running ODAS and on the machine receiving audio to see if any of them is saturated. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[GodCed]

That was a feature under developpement in ODAS a few months ago, but the granular branch doesn’t exist anymore. I am not aware of the current state of this feature.

You could try to get more info on the ODAS repository.

[gary0351]

I can reduce the search volume of the mics to +30 and -20 deg to see how that affects processor loading.

I will also look at the ODAS repository and your papers again.

The potential sources feed the separated sources processing. Are the postfiltered sources used to generate the tracked sources info? If disabled, does it cause other things to fail?

Thanks for your help. Gary

From: Cédric Godin Sent: Monday, January 7, 2019 10:32 PM To: introlab/odas_web Cc: gary0351; Author Subject: Re: [introlab/odas_web] Problem running in windows 10 and linux (#22)

That was a feature under developpement in ODAS a few months ago, but the granular branch doesn’t exist anymore. I am not aware of the current state of this feature. You could try to get more info on the ODAS repository. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[GodCed]

The processing flow is as follow:

• Potential feeds tracking
• Tracking and audio feeds separation
• Separation and tracking feeds post filtering

Post filtering is only applied to the separated audio to improve separation. I can be disabled if the resulting audio stream is not used. Tracking is based solely on the potential sources.

[gary0351]

Thanks for the info. Really appreciate your help, Gary

From: Cédric Godin Sent: Tuesday, January 8, 2019 10:15 AM To: introlab/odas_web Cc: gary0351; Author Subject: Re: [introlab/odas_web] Problem running in windows 10 and linux (#22)

The processing flow is as follow: • Potential feeds tracking • Tracking and audio feeds separation • Separation and tracking feeds post filtering Post filtering is only applied to the separated audio to improve separation. I can be disabled if the resulting audio stream is not used. Tracking is based solely on the potential sources. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[GodCed]

My pleasure. Feel free to open another issue if you experience other difficulties.