TypeError: 'NoneType' object is not iterable, when using bilm_encoder

[mingwei82]

Hi,

I'm learning how to use the gluon-nlp package to learn nlp development step-by-step. I've decoupled parts of the model-zoo language model, where I replaced the encoder layer with my own RNN (e.g. instead of the standard or awd lstm, try doing the encoder layer by myself).

However, when I tried to replace the encoder layer (2-layer LSTM) with the bilm encoder block, I get an error that's in the hybrid forward itself that I don't understand (can't find much help or guidance on how to use the bilm encoder block, https://gluon-nlp.mxnet.io/_modules/gluonnlp/model/bilm_encoder.html)

Error excerpt:

<ipython-input-183-85eeb2083f33> in hybrid_forward(self, F, data, valid_length)
     43         #print(embedded.shape)
---> 44         encoded = self.encoder(embedded)  # Shape(T, N, C)
     45 
     46         print(encoded.shape)

~/anaconda3/envs/mxnet_nlp_p36/lib/python3.6/site-packages/mxnet/gluon/block.py in __call__(self, *args)
    538             hook(self, args)
    539 
--> 540         out = self.forward(*args)
    541 
    542         for hook in self._forward_hooks.values():

~/anaconda3/envs/mxnet_nlp_p36/lib/python3.6/site-packages/mxnet/gluon/block.py in forward(self, x, *args)
    915                     params = {i: j.data(ctx) for i, j in self._reg_params.items()}
    916 
--> 917                 return self.hybrid_forward(ndarray, x, *args, **params)
    918 
    919         assert isinstance(x, Symbol), \

~/anaconda3/envs/mxnet_nlp_p36/lib/python3.6/site-packages/gluonnlp/model/bilm_encoder.py in hybrid_forward(self, F, inputs, states, mask)
    163             which has the same structure with *states[1]*.
    164         """
--> 165         states_forward, states_backward = states
    166         if mask is not None:
    167             sequence_length = mask.sum(axis=1)

TypeError: 'NoneType' object is not iterable

Thanks Mike

[szha]

Hi Mike. Although the API had a default value None for the states field, it's actually required. You can get the value by calling the begin_state function, similar to what's done in the ELMo tutorial.

We will need to add the logic to automatically call begin_state in the hybrid_forward in order to make the None value for state working.