fastai / courses

fast.ai Courses
Apache License 2.0
5.61k stars 2.74k forks source link

Translate.ipynb =>> load_array in load_glove doesn't work! #218

Open Harsh2040 opened 6 years ago

Harsh2040 commented 6 years ago

FileNotFoundError: [Errno 2] No such file or directory: [path to the file]/meta/sizes

Hi, I was struggling to generate the glove.dat file from glove.txt file but I found that one function in the utils2.py is missing which is get_glove.

if you are facing the above problem while running the translate.ipynb file you can add the get_glove function in the utils2.py which I have provided below and call get_glove function before load_glove function.

def get_glove(name, path, res_path):
    with open(path+ 'glove.' + name + '.txt', 'r', encoding='utf8') as f: lines = [line.split() for line in f]
    words = [d[0] for d in lines]
    vecs = np.stack(np.array(d[1:], dtype=np.float32) for d in lines)
    wordidx = {o:i for i,o in enumerate(words)}
    save_array(res_path+name+'.dat', vecs)
    pickle.dump(words, open(res_path+name+'_words.pkl','wb'))
    pickle.dump(wordidx, open(res_path+name+'_idx.pkl','wb'))