[data request] WikiText-103

tensorflow / datasets

TFDS is a collection of datasets ready to use with TensorFlow, Jax, ...

https://www.tensorflow.org/datasets

Apache License 2.0

4.28k stars 1.53k forks source link

[data request] WikiText-103 #17

Open rsepassi opened 5 years ago

rsepassi commented 5 years ago

Name of dataset: WikiText-103
URL of dataset: https://blog.einstein.ai/the-wikitext-long-term-dependency-language-modeling-dataset/
License of dataset: CC BY-SA 3.0 Unported
Short description of dataset and use case(s): The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia.

Folks who would also like to see this dataset in tensorflow/datasets, please +1/thumbs-up so the developers can know which requests to prioritize.

cuent commented 5 years ago

Hey @rsepassi, I'd like to take this issue. Could you please assign the issue to me?

rsepassi commented 5 years ago

Great! Let me know when you've accepted the invite, and then I'll assign it.

cuent commented 5 years ago

Done, thanks!

dhirensr commented 4 years ago

@rsepassi : i would like to take this task. however in this dataset there are 3-4 variations like word level and Raw / character level. which version dataset should be download with this? could you elaborate a bit?

jmr commented 3 years ago

@dhirensr Based on what other datasets do, I'm guessing you'd download both then have wikitext103/word and wikitext103/raw.

georgedahl commented 1 year ago

Is this still being worked on? @cuent are you still intending to complete this?