Add an XLMRobertaMaskedLM task model - Githubissues

keras-team / keras-hub

Pretrained model hub for Keras 3

Apache License 2.0

765 stars 230 forks source link

Add an XLMRobertaMaskedLM task model #720

Closed mattdangerw closed 1 year ago

mattdangerw commented 1 year ago

In https://github.com/keras-team/keras-nlp/pull/653 we added a masked language modeling task for RoBERTa. We can make a similar change for the XLMRoberta model.

[ ] Update XLMRobertaTokenizer to expect a mask token.
[ ] Add a XLMRobertaMaskedLMPreprocessor preprocessor layer and tests.
[ ] Add a XLMRobertaMaskedLM task model and tests.
[ ] Update keras_nlp/models/__init__.py to export XLMRobertaMaskedLM and XLMRobertaMaskedLMPreprocessor.

soma2000-lang commented 1 year ago

I would love to work on this issue!

soma2000-lang commented 1 year ago

/assign

shivance commented 1 year ago

Hi @soma2000-lang are you still working on it? if not then I would take this one up !

soma2000-lang commented 1 year ago

@shivance yes ,I had also opened a pr for it,but had to close it for some reason.Will reopen another one.

jayam30 commented 1 year ago

Hey! I would like to take this issue up, if others are not @soma2000-lang

soma2000-lang commented 1 year ago

@jayam30 already opened a pr

shivance commented 1 year ago

Hi @mattdangerw , it's already been 1month+ since this issue has gone stale. Please assign me this issue, I'm already having a crack at it to see how challenging it is. So far the tests pass in local.

If you allow, I could open a PR. Thanks !

soma2000-lang commented 1 year ago

@shivance I am working on it but the tests are failing currently ,working on that.