Nbickford
commented
3 years ago Hi Michael! Sorry for the long wait; I don't have a fix for this yet and I think I need to do some research, but here's a sort of tree diagram of my thoughts at the moment.
- I think some of the artifacts might come from the windowing method; I wouldn't be surprised if there's a bug here
- Currently, we window -> modify FFT coefficients -> overlap and add tiles
- But if modifying the FFT coefficients causes the start and end of the tile to be != 0, then that's a recipe for clicking at the end of each tile
- DCT-then-window-then-add would get around this
- So would windowing before and after FFT coefficients, then adding with weights other than 1 per tile
- Or maybe applying the square root of the window before and after? But that's probably not optimal
- Could look into overlap-and-add implementations and phase vocoder implementations and see what the correct approach is
- What if we switched to DCT?
- Weiner deconvolution is supposed to be optimal with respect to squared error...
- But we make the assumption that the noise is proportional to the noisy signal which might mess everything up
- DCT makes sense intuitively (it does the same sort of frequency band separation as FFT), though I haven't worked through the math
- DCT would also have benefit that I don't need to introduce complex numbers
- Better approaches are definitely possible - neural network techniques Xiph.org and Mozilla's RNNoise do really well
- Way to check to see if it's improved might be to measure the squared error of each approach, comparing the squared difference of a known pure signal, and the result of denoising that signal with noise added.
- What if we split the problem?
- This project has two goals - to be a learning resource for JSFX, and to be a denoiser
- However, maybe it would be more effective to split it into two projects, each of which could maybe handle its goal better?
- I.e. maybe one project that's like a cookbook of shorter JSFX techniques, and the other which is a high-quality denoiser like RNNoise (if it's possible for me to work on that)
- This then might be more of an archive? But would still be important to fix windowing issues in case that winds up in other FFT-based audio effects.
micsthepick
I've messed around with implementing this algorithm with the MDCT (built into JSFX), and it seems to have some interesting side effects. For one, it seems to greatly reduce the amount of ringing/echo from the processing, and it also seems to be significantly more effective for a given value of k squared.
I think it may be interesting to include a second copy that uses the MDCT (perhaps not replace the existing version, since the MDCT has a much smaller maximum frame size)
The reason that it is so effective may also be in part (but I suspect that this is not the whole story) because of the moving average in my version. (Each time the noise buffer is written to, it takes the weighted average of 3 times the old value and one times the new recorded value), so even if you don't include a MDCT version, I think it helps to get a more accurate noise profile by taking a proper average (perhaps even a moving average, so that all samples are weighted evenly, rather than more recent ones with greater weight)