search

xanguera / BeamformIt

BeamformIt acoustic beamforming software
353 stars 111 forks source link

question about real-time performance #17

Closed jingxuan9862 closed 4 years ago

jingxuan9862 commented 7 years ago

Hi , when I use the BeamformIt to process my rocodings ,it preforms well ,but there is a doubt that can the BeamformIt work realtime?

xanguera commented 7 years ago

Hi, I suspect you mean "online",right? IT currently does not work online because it performs some postprocessing to the TDOA values to improve them. If you refactored the code and ignored this postprocessing and some preprocessing it could run online.

On Mon, May 1, 2017 at 9:22 AM, jingxuan9862 notifications@github.com wrote:

Hi , when I use the BeamformIt to process my rocodings ,it preforms well ,but there is a doubt that can the BeamformIt work realtime?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/xanguera/BeamformIt/issues/17, or mute the thread https://github.com/notifications/unsubscribe-auth/AJE6_H4WXZARf-Vo0YaoDVznUy4rUahhks5r1ZZIgaJpZM4NM6FA .

jingxuan9862 commented 7 years ago

ok ,thank you.

kukzile commented 6 years ago

The compilation takes about 15 seconds. Can that be reduced if we want to use this in real time? By real time i mean,recording data, then processing it (beamform) and passing it to speech recognition algorithm.

xanguera commented 6 years ago

I am sure you could. The original code was not coded for speed, but as a research project. One quick thing you could do is disable the postprocessing of TDOA delays, but this will deteriorate a bit the results...

On Wed, Jul 18, 2018 at 12:57 PM, kk2496 notifications@github.com wrote:

The compilation takes about 15 seconds. Can that be reduced if we want to use this in real time? By real time i mean,recording data, then processing it (beamform) and passing it to speech recognition algorithm.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/xanguera/BeamformIt/issues/17#issuecomment-405905577, or mute the thread https://github.com/notifications/unsubscribe-auth/AJE6_MumsKsqklL8z6t7Xu6a-rEt8RTBks5uHyKjgaJpZM4NM6FA .

dyustc commented 1 year ago

Hi, I am also consider using it in real-time, there is one small issue bothering me, I saw the default setting of scroll(shift) length of audio is 250ms, or the window length is 500ms, is this a lot to bear as for real time communication? I am wondering if a much smaller shift could be put to use and still have a decent beamforming effect.

xanguera commented 1 year ago

Yes, you can try with smaller window and shift. The performance might degrade though.

On Wed, 14 Jun 2023 at 08:59, yzdai @.***> wrote:

Hi, I am also consider using it in real-time, there is one small issue bothering me, I saw the default setting of scroll(shift) length of audio is 250ms, or the window length is 500ms, is this a lot to bear as for real time communication? I am wondering if a much smaller shift could be put to use and still have a decent beamforming effect.

— Reply to this email directly, view it on GitHub https://github.com/xanguera/BeamformIt/issues/17#issuecomment-1590675131, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACITV7DSIU3NZVDSQ2A2AL3XLFVMZANCNFSM4DJTUFAA . You are receiving this because you modified the open/close state.Message ID: @.***>

dyustc commented 1 year ago

Thanks for the quick reply. I tested it with a window length of 20ms, and scroll length of 10ms, much smaller than the default ones, there is a little bit degradation, but not much.

But I also found that if the reference channel number is calculated adaptively instead of using the whole picture, the performance is degraded more. Anyway, I am thinking using this beamformer as a frontend for a more solid denoised output of a neural network. Currently with the default setting, it works fine. Maybe as I change the setting more, there will be more findings.

xanguera commented 1 year ago

If you know which channel should be the best, then skip the reference channel calculation. I believe there was an option for that. I agree that with such small window you will not have enough info to select a good reference adaptively. Another solution (if you do not know the best channel a priori) would be ro tely a lot more on the history so that the reference channel stabilizes over time (i.e. it does not use so much of the current info, but more of the aggregate prior). There is nothing there that does that, you would need to built it.

Xavier

On Fri, 16 Jun 2023 at 10:27, yzdai @.***> wrote:

Thanks for the quick reply. I tested it with a window length of 20ms, and scroll length of 10ms, much smaller than the default ones, there is a little bit degradation, but not much.

But I also found that if the reference channel number is calculated adaptively instead of using the whole picture, the performance is degraded more. Anyway, I am thinking using this beamformer as a frontend for a more solid denoised output of a neural network. Currently with the default setting, it works fine. Maybe as I change the setting more, there will be more findings.

— Reply to this email directly, view it on GitHub https://github.com/xanguera/BeamformIt/issues/17#issuecomment-1594391964, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACITV7AAYXBSPZ25OT3K3F3XLQRIBANCNFSM4DJTUFAA . You are receiving this because you modified the open/close state.Message ID: @.***>