search

seongminp / transformers-into-vaes

Code for "Finetuning Pretrained Transformers into Variational Autoencoders"
https://aclanthology.org/2021.insights-1.5
36 stars 11 forks source link

KL Thresholding #5

Closed fhrzn closed 2 years ago

fhrzn commented 2 years ago

Hello again! Thank you for the great code. As stated in the paper, there is a KL Thresholding technique. But, I feel lost looking for this function in the code. Could you please point out where I could find it?

Thank you very much!

seongminp commented 2 years ago

Hi! T5VAE has a attribute min_z that it uses to enforce KL thresholding on this line.

fhrzn commented 2 years ago

Got it, thank you very much! 👍