EOFError: Compressed file ended before the end-of-stream marker was reached

harvardnlp / annotated-transformer

An annotated implementation of the Transformer paper.

http://nlp.seas.harvard.edu/annotated-transformer

MIT License

5.68k stars 1.23k forks source link

EOFError: Compressed file ended before the end-of-stream marker was reached #33

Closed suenpun closed 5 years ago

suenpun commented 5 years ago

when split dataset with this following line ,the error happens:

train, val, test = datasets.IWSLT.splits(
    exts=('.de', '.en'), fields=(SRC, TGT),
    filter_pred=lambda x: len(vars(x)['src']) <= MAX_LEN and
                          len(vars(x)['trg']) <= MAX_LEN)

I think that's mostly due to the versions' incompatible my versions(actually there is not a requirements file in the repo,I'll be gratefull if someone add it ):

torch==0.3.0.post4
torchtext==0.2.3
spacy==2.0.10
torchvision==0.2.1

thank u!

suenpun commented 5 years ago

Error is gone after the files in .data/* are deleted and rerun