search

lifeiteng / vall-e

PyTorch implementation of VALL-E(Zero-Shot Text-To-Speech), Reproduced Demo https://lifeiteng.github.io/valle/index.html
https://lifeiteng.github.io/valle/index.html
Apache License 2.0
2.03k stars 319 forks source link

Question about AR model loss #156

Closed asr-pub closed 1 year ago

asr-pub commented 1 year ago

Hello, I have a question about the CE loss, Why use the sum mode of CE loss ?

    def forward(
        self,
        x: torch.Tensor,
        x_lens: torch.Tensor,
        y: Union[torch.Tensor, PromptedFeatures],
        y_lens: Union[torch.Tensor, PromptedFeatures],
        reduction: str = "sum",
        train_stage: int = 0,
        **kwargs,
    ) -> Tuple[torch.Tensor, Union[torch.Tensor, None]]:
  ...
  total_loss = F.cross_entropy(logits, targets, reduction=reduction)
  ...
lifeiteng commented 1 year ago

@asr-pub you can try mean