-
Hi ! As you know TAR has some constraints for data streaming. While it is optimized for buffering, the files in the TAR archive **need to be streamed in order**. It means that we can't choose which fi…
-
## Environment info
- `transformers` version: 4.3.0.dev0
- Platform: Ubuntu
- Python version: 3.6.12
- PyTorch version : 1.7.1
- Using GPU in script?: Y
- Using distributed or parallel set-up …
-
Sik-Ho Tsang. [Review: Representation Learning with Contrastive Predictive Coding (CPC/CPCv1)](https://sh-tsang.medium.com/review-representation-learning-with-contrastive-predictive-coding-cpc-cpcv1-8…
-
Thanks for your wonderful work!
I am very interested in your work and try to extend the ideas to other tasks.
I wonder when the rendering instructions will be available?
-
Initialization of the client:
```js
// @flow
import {
ApolloClient
} from 'apollo-client';
import {
HttpLink
} from 'apollo-link-http';
import {
InMemoryCache,
IntrospectionFrag…
-
## ❓ Questions & Help
I need to compare my research against distilBERT as a baseline for a paper in progress. I went through your publication and found that you don't report accuracies on the glu…
smr97 updated
2 years ago
-
Using a dataset with a single 'text' field and a fast tokenizer in a jupyter notebook.
```
def tokenizer_fn(example):
return tokenizer.batch_encode_plus(example['text'])
ds_tokenized = te…
-
Hi authors,
Thanks for sharing the repo. It seems that the original version of the dataset is unavailable online. If you have a copy, can you make it available for download? Otherwise, please sugge…
-
I found that in the dataset description, we can `Use process_data.py for pre-processing wikipedia/bookcorpus datasets into a single text file.`
What if I want to process these two datasets at the s…
-
Hello, I am very insterested in trying to pre-train the langage model from scratch.
But I am not sure about the ratio between validpref and trainpref in actual pretraining?
For example, BERT, the co…