Not able to load encoder

[008karan]

getting error while loading encoder: m3.load_encoder(f'adam1_enc')

RuntimeError                              Traceback (most recent call last)
~/anaconda2/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
    513                 try:
--> 514                     own_state[name].copy_(param)
    515                 except Exception:

RuntimeError: invalid argument 2: sizes do not match at /opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/generic/THCTensorCopy.c:51

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
<ipython-input-154-c173cedbc36b> in <module>
----> 1 m3.load_encoder(f'/home/gamut/Downloads/fastai/courses/dl1/data/hindilm/models/adam1_enc')

~/Downloads/fastai/courses/dl1/fastai/nlp.py in load_encoder(self, name)
    164     def save_encoder(self, name): save_model(self.model[0], self.get_model_path(name))
    165 
--> 166     def load_encoder(self, name): load_model(self.model[0], self.get_model_path(name))
    167 
    168 

~/Downloads/fastai/courses/dl1/fastai/torch_imports.py in load_model(m, p)
     38             if n+'_raw' not in sd: sd[n+'_raw'] = sd[n]
     39             del sd[n]
---> 40     m.load_state_dict(sd)
     41 
     42 def load_pre(pre, f, fn):

~/anaconda2/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
    517                                        'whose dimensions in the model are {} and '
    518                                        'whose dimensions in the checkpoint are {}.'
--> 519                                        .format(name, own_state[name].size(), param.size()))
    520             elif strict:
    521                 raise KeyError('unexpected key "{}" in state_dict'

RuntimeError: While copying the parameter named encoder.weight, whose dimensions in the model are torch.Size([67979, 300]) and whose dimensions in the checkpoint are torch.Size([25704, 300]).

I was able reproduce the notebook given by you but again getting the same error. Not facing this issue when I load encoder from lesson 4 notebook which is for imdb data. Can you look through it, otherwise people will not be able to use your pretrained LM

[NirantK]

Could you please share the steps before this line?

It seems like you are trying a different model in some manner, this is indicated by the difference in dimensions:

RuntimeError: While copying the parameter named encoder.weight, whose dimensions in the model are torch.Size([67979, 300]) and whose dimensions in the checkpoint are torch.Size([25704, 300]).

i.e. [67979, 300] vs [25704, 300]. The Hindi LM model expects the latter

[008karan]

I am following lesson 4 notebook from sentiment analysis part. which starts with loading vocab file. TEXT = pickle.load(open(f'{PATH}/TEXT.pkl','rb')) Then I defined my custom dataset using lang_model-arxiv.ipynb. Then I was able get data in required format.

t = splits[0].examples[111]
t.label, ' '.join(t.text[:19])

output:

('hotel', 'हम्पी में 3 दिसंबर से 3 दिन 2 रात के लिए एक कमरा बुक करें')

which is working same as shown in lesson 4 Then followed the rest of the notebook. But got stuck when loading encoder.

md2 = TextData.from_splits(PATH, splits, bs)
m3 = md2.get_model(opt_fn, 1500, bptt, emb_sz=em_sz, n_hid=nh, n_layers=nl, 
dropout=0.1, dropouti=0.4, wdrop=0.5, dropoute=0.05, dropouth=0.3)
m3.reg_fn = partial(seq2seq_reg, alpha=2, beta=1)

m3.load_encoder(f'adam1_enc')

initially I used vocab and encoder provided by you but due above error I trained my language model by following your notebook but again got stuck at load_encoder step.

[008karan]

anything yet

[NirantK]

Hey @008karan, as a solo dev - I don't have the bandwidth to support this older version while building out newer version.

If you'd be so kind to have some patience, the newer version will have an end to end demo.

[008karan]

If possible can you just check that are you able to load encoder or not. When are you going to present newer version?

[ankit201]

Could you please share the steps before this line?

It seems like you are trying a different model in some manner, this is indicated by the difference in dimensions:

RuntimeError: While copying the parameter named encoder.weight, whose dimensions in the model are torch.Size([67979, 300]) and whose dimensions in the checkpoint are torch.Size([25704, 300]).

i.e. [67979, 300] vs [25704, 300]. The Hindi LM model expects the latter

Did you find the solution to it ? @NirantK