use of NoiseBlock and depthwise?

hubertsiuzdak / snac

Multi-Scale Neural Audio Codec (SNAC) compresses audio into discrete codes at a low bitrate

https://hubertsiuzdak.github.io/snac/

MIT License

439 stars 26 forks source link

Closed listener17 closed 7 months ago

listener17 commented 8 months ago

Hi!

Interesting work. But I don't understand the following changes:

I think you're doing "depthwise" for reducing the number the parameters. Correct?
What's the use of NoiseBlock in the decoder?

hubertsiuzdak commented 7 months ago

Hello!

Depthwise convolutions not only reduced the parameter count, but also helped me optimize the model. After replacing most of the convolutions with DW, I could use higher learning rates without the model collapsing. Surprisingly, the perceived quality was higher!
Injecting noise into the decoder improved my models. I believe it's helpful to introduce some stochasticity