search

p0p4k / vits2_pytorch

unofficial vits2-TTS implementation in pytorch
https://arxiv.org/abs/2307.16430
MIT License
471 stars 84 forks source link

log_w_ not detach #21

Closed hdmjdp closed 1 year ago

hdmjdp commented 1 year ago

https://github.com/p0p4k/vits2_pytorch/blob/cf513a71e07aed48448e582eaecfaef2a2b8d6b6/train_ms.py#L265

true duraion should not detach

p0p4k commented 1 year ago

https://github.com/p0p4k/vits2_pytorch/blob/cf513a71e07aed48448e582eaecfaef2a2b8d6b6/train_ms.py#L265

true duraion should not detach

Why not? It is not supposed to carry any gradients to the main net_g model. Also, I believe that logw itself doesn't have any gradients, since it is calculated in MAS attn. So, detach or not should not make difference. Am I missing something?

hdmjdp commented 1 year ago

https://github.com/p0p4k/vits2_pytorch/blob/cf513a71e07aed48448e582eaecfaef2a2b8d6b6/train_ms.py#L265

true duraion should not detach

Why not? It is not supposed to carry any gradients to the main net_g model. Also, I believe that logw itself doesn't have any gradients, since it is calculated in MAS attn. So, detach or not should not make difference. Am I missing something?

As you said, detaching the true label has no meaning.