TypeError: index must be integers when adding an EmbeddingLayer to a network

[NenadZivic]

Hi, I'm trying to add an EmbeddingLayer to my network (a simple MLP net). Here is the complete code, as it is not very long:

from lasagne import layers
from lasagne.updates import nesterov_momentum
from nolearn.lasagne import NeuralNet
import numpy as np
from lasagne.nonlinearities import softmax
from sklearn.preprocessing import LabelEncoder
from sklearn.utils import shuffle

def load():
    f = open("output_sone.txt", "r")
    lines = f.readlines()
    f.close()
    X = []
    for line in lines:
    #    X.append(np.array([np.float32(y) for y in line.strip().split(' ')],dtype=np.float32))
    #    X.append(np.array([np.int32(y) for y in line.strip().split(' ')],dtype=np.int32))
        X.append(np.array([int(y) for y in line.strip().split(' ')],dtype=int))

    f = open("output_labels_sone.txt", "r")
    lines = f.readlines()
    f.close()
    #y = [np.int32(int(x.strip())) for x in lines]
    y = [int(int(x.strip())) for x in lines]

    #print(np.array(y))
    #use only for classification budz
    #encoder = LabelEncoder()
    #y = encoder.fit_transform(y).astype(np.int32)
    #print(y)

    X = np.array(X).reshape(-1, 20, 100, 1)
    #y = np.array(y).reshape(-1, 1)

    X, y = np.array(X), np.array(y)
    X, y = shuffle(X, y, random_state=42)

    print(X[1,1,1,0], type(y[1]))

    print(X.shape, y[1])

    return X, y

early_churn_pred_net_1 = NeuralNet(
    layers=[
       ('input', layers.InputLayer),
        ('embedding', layers.EmbeddingLayer),
        ('hidden', layers.DenseLayer),
        ('dropout2', layers.DropoutLayer),
        ('hidden2', layers.DenseLayer),
        ('output', layers.DenseLayer),
        ],
    #nisam siguran oko input shape
    #None znaci variable batch size
    #1 treba da znaci da je jedan kanal
    #100 znaci da je 1D konvolucija (nadam se da ne mora jos jedan 1 kao druga D)

    input_shape=(None, 20, 100, 1),

    #broj filtara (feature mapova) kao i njihova velicina prilicno proizvoljni

    hidden_num_units = 100,
    hidden2_num_units = 30,

    embedding_input_size = 2,
    embedding_output_size = 100,
    #embedding_W = np.random.rand(2, 100),

    #dropout1_p = 0.8,
    dropout2_p = 0.5,

    output_num_units=2,
    #output_nonlinearity=None,
    output_nonlinearity=softmax,
    #trenutno bez pametnog update-ovanja learning rate-a i momenta

    # optimization method:
    update=nesterov_momentum,
    update_learning_rate=0.01,
    update_momentum=0.9,
    #use_label_encoder=False,

    regression=False,
    eval_size = 0.2,
    max_epochs=100,
    verbose=1,
    )

X, y = load()
early_churn_pred_net_1.fit(X, y)

And I get this error:

Traceback (most recent call last):
  File "najmanja_mreza.py", line 91, in <module>
    early_churn_pred_net_1.fit(X, y)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nolearn/lasagne/base.py", line 442, in fit
    self.initialize()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nolearn/lasagne/base.py", line 288, in initialize
    self.y_tensor_type,
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nolearn/lasagne/base.py", line 395, in _create_iter_funcs
    layers, target=y_batch, **objective_kw)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nolearn/lasagne/base.py", line 146, in objective
    output_layer, deterministic=deterministic, **get_output_kw)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Lasagne-0.2.dev1-py3.4.egg/lasagne/layers/helper.py", line 185, in get_output
    all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Lasagne-0.2.dev1-py3.4.egg/lasagne/layers/embedding.py", line 68, in get_output_for
    return self.W[input]
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/sandbox/cuda/var.py", line 142, in __getitem__
    return _operators.__getitem__(self, *args)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/var.py", line 474, in __getitem__
    return self.take(args[axis], axis)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/var.py", line 506, in take
    return theano.tensor.subtensor.take(self, indices, axis, mode)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/subtensor.py", line 2290, in take
    return take(a, indices.flatten(), axis, mode).reshape(shape, ndim)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/subtensor.py", line 2268, in take
    return advanced_subtensor1(a, indices)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/gof/op.py", line 509, in __call__
    node = self.make_node(*inputs, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/subtensor.py", line 1608, in make_node
    raise TypeError('index must be integers')
TypeError: index must be integers

Has anyone had the same error, or I am doing something wrong? If I am, could you please provide me with a simple example of how it should be done, as I didn't manage to find any example online.

Thanks, Nenad

[justiceamoh]

I am getting the same error too when i try to use the embedding layer. My network looks like this:

layers=[
        (InputLayer,     {'shape': (None,NUM_FEATURES)}),
        (EmbeddingLayer, {'input_size':  NUM_FEATURES, 'output_size': 750}),
        (DenseLayer,     {'num_units':  750, 'nonlinearity':relu}),
        (DropoutLayer,   {'p':0.5}),
        (DenseLayer,     {'num_units':  500, 'nonlinearity':relu}),
        (DropoutLayer,   {'p':0.5}),
        (DenseLayer,     {'num_units':  250}),                     
        (DenseLayer,     {'num_units':NUM_CLASSES, 'nonlinearity':softmax}),
    ]

net = NeuralNet(
        layers=layers,
        max_epochs=5,
        update=nesterov_momentum,
        update_learning_rate=0.001,
        update_momentum=0.9,
        train_split=TrainSplit(eval_size=0.25),
        verbose=1,
    )

did you figure out what was causing this error?

[BenjaminBossan]

It is definitely possible, though I don't have working code here right now. My first attempt would be to set the input_var parameter of the InputLayer to be integer type (for instance T.imatrix).

@NenadZivic Why is your input 4D?

[dnouri]

InputLayer's input_var is by default set to:

input_var_type = T.TensorType(theano.config.floatX, [False] * ndim)

Explicitly passing an input var should fix your issue. E.g.:

InputLayer((None, NUM_FEATURES), input_var=T.imatrix())

[shantanudev]

Yup, this fixed the issue. Thank you @dnouri

[dapeng2018]

it is the problem, thx @dnouri