-
In https://github.com/keras-team/keras-nlp/pull/653 we added a masked language modeling task for RoBERTa. We can make a similar change for the `XLMRoberta` model.
* [ ] Update `XLMRobertaTokenizer`…
-
### Summary
Unable to use a SimpleTransformes ClassificationModel in streamlit
### Setup and code
I've got a simple demo.py file in which I first load and cache a SimpleTransformers `Classifi…
-
@abheesht17 @mattdangerw I think this needs to be fixed
-
### This issue is part of our **Great Code Cleanup 2022**. If you're interested in helping out, take a look at [this thread](https://twitter.com/carrigmat/status/1502319813510766599), or come [join us…
-
Hello, while researching Longformer conversion, I came across your repo.
In `convert_roberta_to_longformer` you do not copy the token_type_embeddings from Roberta source model to the new Longformer…
-
The work on this is excellent. I've been using for fine-tuning sentiment / topic models with great success!
One very nitpicky thing I noticed in the repo is that the README.md contains a link to ht…
-
We need to simplify our model docstring so they are more easily understandable, and we can start by porting the changes in https://github.com/keras-team/keras-nlp/pull/843 to all other models.
Let'…
-
1. `roberta_models.py` -> `roberta_backbone.py`
2. `roberta_models_test.py` -> `roberta_backbone_test.py`
3. `Roberta` -> `RobertaBackbone`
Same idea for `XLMRoberta`.
See #526 for more backgr…
-
I am unsure as to what weights are updated when training the following pipeline:
As a TransformerDocumentEmbeddings I use a huggingface model such as 'xlm-roberta-base'.
A CSVClassificationCorpus is…
-
**Issue:**Conversion works well with the dynamic axis 1 i.e. batch size, While trying conversion with both dynamic axes i.e. batch size & input text sequence length for the Huggingface XLMRoberta base…