search

jadore801120 / attention-is-all-you-need-pytorch

A PyTorch implementation of the Transformer model in "Attention is All You Need".
MIT License
8.78k stars 1.97k forks source link

preprocess ERROR #135

Open JingsenZhang opened 4 years ago

JingsenZhang commented 4 years ago

Who can tell me where this path can be changed? I have installed this package, but I don't know where to change the path below. Thank you!!! image

jadore801120 commented 4 years ago

@JingsenZhang That is interesting. I haven't test my code under Windows system. The extension name (.de_core_news_sm) is different from mine (.de). Please check if the corpus files are under the data folder (.data\multi30k\).

To set your own path to your corpus files, set up the parameters root and exts in the splits function call, which indicate the file dir + file basename and file extension name respectively. doc

Hope this helps. Feel free to ask if you have other problems.

Best, Yu-Hsiang

JingsenZhang commented 4 years ago

@jadore801120 Thank you! The shortcut was not available when I installed 'en' and 'de' on Windows, like this image When i use 'en' in code,it says it is not available.So I try to replace 'en' in the code with en_core_web_sm, however,under the data folder ,it is 'train.en' not 'train.en_core_web_sm' image It feels like a contradiction.

jadore801120 commented 4 years ago

@JingsenZhang I install the spacy model with python -m spacy download en; python -m spacy download de. It seems that you install with python -m spacy download en_core_web_sm. In this case, the extension name and the spacy model name are inconsistent. Feel free to split it into 2 different variables (don't just use opt.lang_src to indicate both file extension name and spacy model). Hope this helps. Yu-Hsiang

JingsenZhang commented 4 years ago

@jadore801120 Thank you for your help !!! And I have performed all the actions you listed. image

How can I use this model, for example to complete a small task or test its effect. Thank you again!