TWOEARS / auditory-front-end

Two!Ears Auditory Model - Auditory front-end module
http://docs.twoears.eu/en/latest/afe/
GNU General Public License v2.0
16 stars 10 forks source link

new procs + complex buffer + fixes to masking #16

Closed kashefy closed 7 years ago

kashefy commented 7 years ago

This PR contains: A. New Processors:

B. Signal classes for Processors above. C. Circular buffer for complex signals D. Fixes for masking spatio temporal signals

@Hardcorehobel, the above changes are actively being used to demo scenarios (specifically estimating the number of sources in scenarios involving multiple sounds). Merging them into master, will make things easier to manage.

ivo--t commented 7 years ago

@Hardcorehobel , if you are unsure about whether to include DUET/STFT into master: please at least merge #9 and #14 , which are included in this one here, but are bugfixes.

hagenw commented 7 years ago

Could you also create a pull request for the documentation of the two new processors.

Hardcorehobel commented 7 years ago

Hey @kashefy, since this pull request introduces two new processors, I think it would be great if you could add two short demo's to the test folder which would illustrates their functionality. You could use the reverberant single talker signal for the DUET. The STFT processor could be compared to the ratemap representation. What do you think?

kashefy commented 7 years ago

Will do. (demo + docs)

Hardcorehobel commented 7 years ago

One remark about DUET: when using the standard approach, a microphone spacing of about 4.5 cm is required for proper source localization. However, it is possible to use the Big Delay DUET for larger distances. Which method did you implement here @kashefy ?

kashefy commented 7 years ago

This is a question for @pmeier82

Hardcorehobel commented 7 years ago

@kashefy : In the function STFTSignal.m, the case 'othersie' of the switch command is suboptimal. Either supply a default value for do_dB or replace the warning with an error. In general, I can't get the STFT plotting routine to work. Did you consider this?

kashefy commented 7 years ago

Agreed. Done.

Hardcorehobel commented 7 years ago

@kashefy So it seems that the STFT processors doesn't have its own plotting routine, right? In addition, I have one question regarding the STFT output. How is it being used? I cannot access its output directly using ... dObj.stft{1}.Data(:) which gives me an error.

hagenw commented 7 years ago

Maybe this is a general problem at the moment. I haven't tested it, but at https://github.com/TWOEARS/TwoEars/issues/6 it was reported that itd = dataObj.itd{1}.Data(:) does also not work.

Hardcorehobel commented 7 years ago

I just checked it: the data access, e.g. dObj.itd{1}.Data(:), does work with R2013b. However, it does NOT work with R2015b, R2016a or R2016b :(. So it seems to be a general problem. @ivo--t any ideas?

pmeier82-zz commented 7 years ago

Hi all,

I used MATLAB 2015a for development.

I tried to tag along the example method implementations from the templateproc.m.

The phase warp issue is valid in principle. but should not be important for the number of sources estimation. It is sufficient to have unique mixture parameters such that sources are separable at all. This becomes a (rather big!) issue for reconstruction and and masking though!

The Big-Delay DUET has not been implemented. The uspampling as well as the tiling approach have a steep resource cost. And cache-space as well as compute time was at a premium.

The most conservative maximum microphone distance is C/sample_rate_hz. If you can make good assumptions about the dominant band of your signal(s) you can replace samplingrate with 2*dominant frequency. For 16kHz sampling rate and C~330m/s that comes out to be on the order of 2cm.

Hth, let me know if I can help with anything!

Hardcorehobel commented 7 years ago

Great, looking good! @pmeier82 are you planning to add a DUET demo as well ?

kashefy commented 7 years ago

DUET demo added. We're only computing up to an intermediate step of the DUET algorithm. which the demo shows next to a ILD/ITD histogram.