search

thesofproject / sof

Sound Open Firmware
Other
541 stars 312 forks source link

[FEATURE] Beamformer #1537

Closed lgirdwood closed 4 years ago

lgirdwood commented 5 years ago

Is your feature request related to a problem? Please describe. Beamformer using array of 4 or more DMICs

Describe the solution you'd like @singalsu will leave this open for you to define.

singalsu commented 5 years ago

Great, I think this will be very useful. I'd propose to start with a fixed time domain beamformer for a 16 kHz DMIC pipeline. Such simple beamformer is well compatible with with possible with other firmware or user space based other processing like AEC and NS.

The filter coefficients would be passed like EQ coefficients from user space. We have now the large data IPC support so this can be done (e.g. 64 coefficients per microphone, 16 bit). There would need to be topology data sets to include for a number of mic count and spacing configurations, e.g. two microphones and 50 mm spacing. A suitable configuration needs to be selected manually per device form factor.

Such beamformer can enhance notebook audio capture into user facing camera direction. There should be noticeable improvement with just 2 microphones. The notebooks have typically mic arrays with quite large spacing so beamforming for 48 kHz generic audio is not really feasible because the array mic spacing at 4-5 cm works best for voice bandwidth. 48 kHz beamforming would of course work too but will require custom arrays with higher mic count.

lgirdwood commented 4 years ago

@singalsu I guess this is on target for v1.6 given your current work ?

lgirdwood commented 4 years ago

@singalsu any update on your PR https://github.com/thesofproject/sof/pull/2925 . UUID now all merged so no blockers.

mengdonglin commented 4 years ago

Closed as PR #2925 has been merged and recorded wav has less background noise with Beamformer on TGL.

wmertens commented 4 years ago

@mengdonglin I had a look at the PR but couldn't see any documentation. Is there somewhere an explanation of how to use this? Thanks :)

lgirdwood commented 4 years ago

@singalsu any comment here on the docs. Are they still WIP ?

singalsu commented 4 years ago

@lgirdwood @wmertens Yep, I'm doing documentation to SOF Docs. I'd expect it take one week or so for the raw content plus one more week to work with language fixes etc.