Open aatkinson opened 2 years ago
I'm also interested in deberta-mt implementation, and there is grey area for unilm implementation for example how we can implement disentangled attention, how did author dealt with relative position bias for incremental model, etc.
Hi, in appendix 4 of the paper you describe results on NLG tasks and a DeBERTa-MT base model pretrained on wikitext-103, using unilm task formulations.
Are you going to release this model and the code for the associated experiments?
Thanks