DebertaV3: is EMD used in generator and discriminator?

microsoft / DeBERTa

The implementation of DeBERTa

MIT License

1.91k stars 216 forks source link

DebertaV3: is EMD used in generator and discriminator? #92

Open stevenliu000 opened 2 years ago

stevenliu000 commented 2 years ago

Impressed by the performance debertaV3, and I'm trying to re-produce debertaV3.

I have a question regarding the model architecture in DebertaV3. Is EMD used in generator when doing MLM task? I see the released code for ReplacedTokenDetectionModel and it seems that Discriminator does not use EMD. Can you comfirm that?

Thanks!