search

keonlee9420 / DiffGAN-TTS

PyTorch Implementation of DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs
MIT License
320 stars 44 forks source link

What does the mlp and Mish function in modules.py do #4

Closed Dyongh613 closed 2 years ago

Dyongh613 commented 2 years ago

self.mlp = nn.Sequential( LinearNorm(residual_channels, residual_channels 4), Mish(), # return x torch.tanh(F.softplus(x)) LinearNorm(residual_channels * 4, residual_channels) )

class Mish(nn.Module): def forward(self, x): return x * torch.tanh(F.softplus(x))

keonlee9420 commented 2 years ago

Hi @qw1260497397 , it is for the embedding of diffusion time steps as described in the paper.

keonlee9420 commented 2 years ago

Close due to inactivity.