Closed melisandeteng closed 4 years ago
we really need to discuss this encoder, it's huge
we really need to discuss this encoder, it's huge
@vict0rsch Actually if you remove the ResBlocks then it's "only" 42M parameters. If you add 1 resblock it goes up to 1000M !
But I agree that the latent space is larger than what we are used to working with
haha let's have a 100B param model like they do in NLP
haha let's have a 100B param model like they do in NLP
GPT-3 is crazy...no more needs for fine-tune Lol
number of resblocks parameter wasn't propagating correctly in the code...
(it's a teeny-tiny pull request, but then we can keep track of where this problem came from)