anonymous-pits / pits

PITS: Variational Pitch Inference for End-to-end Pitch-controllable TTS without External Pitch Predictor

https://anonymous-pits.github.io/pits/

MIT License

274 stars 34 forks source link

Question about vq in the paper #11

Open taalua opened 1 year ago

taalua commented 1 year ago

Hi, thank you for your work.

I have a question regarding VQ experiment in the paper.

z_spec_q_st, z_spec_q

did you use z_spec_q_st instead of z_spec ? what happen with the indices z_spec_q ? regarding the loss, did you simply add the commitment and vq loss to the overall loss (loss_gen_all)?

Thank you.

anonymous-pits commented 1 year ago

For now VQ implementation is not uploaded and some parts are remained. Our VQ implementation is adopted from this repository.

z_spec_q_st gives gradient to z_spec, and z_spec_q does not give gradient. Thus, to build VQ-VAE, z_spec_q_st is needed.