-
on the examples, it loads bert-base model and do some tasks, the paper says that it will fix the parameters of bert and only update the parameters of our tasks, but i find that it seems not fix param…
-
First up, thanks for the great work and releasing the code!
I'm trying to repro the baselines from the code and it works like a charm for Q->A and QA->R tasks, but I don't see any code for Q->AR ta…
-
I just trained a simple NER tagger (with only french fasttext word embeddings, and no contextual string embeddings).
The model is the following:
```python
SequenceTagger(
(embeddings): Stacked…
mhham updated
5 years ago
-
Two quick related enhancement ideas:
1) Retrieve the actual text of the tokens, alongside their embedding. It's currently a little hard to debug why a string didn't get classified correctly when I ca…
-
Investigate where transformers are suitable for contextual string embeddings.
A working example of language model training is here:
https://github.com/blythed/lm
-
Or if your could offer a convinent way to train it, it'll be helpful! Since the training is costly, and I don't exactly have a way to verify it quickly.
Thanks!
-
Create a model that learns how to extract original dot products from contextualized embeddings (after applying LSTM)
-
Hello,
The dimensionality of the language model you use in LSTM is 2048, that means 4096 for a single word contextual embeddings, have you tried the different dimensionalities? And do you have the …
-
I am attempting to train model on AVA myself and faced very low quality of predictions. Digging further, I found predictions to be very strange and started to investigate pretrained models and asking …
hcl14 updated
5 years ago
-
I am getting a ResourceExhaustedError: OOM - When Doing an embedding usinfg google's trasformer Architecture which embeds the text into a 512 dimensional vectors.
The data I'm trying to embed has 5…