Creating Model variable throws "AttributeError: 'Tensor' object has no attribute '_keras_history'"

[aiwabdn]

Hello all,

setup: Keras 2.0, Tensorflow 1.0, Keras-contrib (using functional api of Keras, but older layer names e.g. Convolution3D). Running on Google Cloud MLEngine with K80 GPU

Problem: The layers are setup properly. All the output shapes match correctly. Inputs and outputs are setup properly. The model takes 5 image inputs and produces 2 outputs. Internally, it takes each image, puts it through layers of residual convolution and updates an LSTM layer. Each image is used recurrently to update the hidden state of the LSTM. The final hidden state of the LSTM is then flattened and put through 2 dense layers to produce 2 output tensors. The problem arises when declaring the Model variable of Keras. We provide the 5 inputs and 2 outputs as suggested in Keras functional api documentation. At this point, we encounter an internal Keras layer error that says AttributeError: 'Tensor' object has no attribute '_keras_history'

Googling for the error shows similar problems with other attributes. As far we have checked, none of them serve our purpose. Any pointers will be highly appreciated.

StackTrace:

Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "main", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/root/.local/lib/python2.7/site-packages/trainer/VAE_GAN.py", line 564, in e_net,g_net,d_net,x_tilde,z_x_mean, z_x_log_sigma_sq, z_x, x_p, d_x, d_x_p, z_p = inference(image_data1,image_data2,image_data3,image_data4, image_data5,real_3d_data) File "/root/.local/lib/python2.7/site-packages/trainer/VAE_GAN.py", line 471, in inference e_net = Model(inputs=[encoder_input1,encoder_input2,encoder_input3,encoder_input4,encoder_input5], outputs=[enc_mean,enc_sigma],name="encoder") File "/root/.local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 87, in wrapper return func(*args, **kwargs) File "/root/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 1705, in init build_map_of_graph(x, finished_nodes, nodes_in_progress) File "/root/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 1695, in build_map_of_graph layer, node_index, tensor_index) File "/root/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 1695, in build_map_of_graph layer, node_index, tensor_index) File "/root/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 1695, in build_map_of_graph layer, node_index, tensor_index) File "/root/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 1695, in build_map_of_graph layer, node_index, tensor_index) File "/root/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 1665, in build_map_of_graph layer, node_index, tensor_index = tensor._keras_history AttributeError: 'Tensor' object has no attribute '_keras_history'

[wolhow123]

same problem here

[fchollet]

The error message will be clearer if you use an up to date version of Keras.

This is a symptom of one of two things:

• your inputs don't come from keras.layers.Input()
• your outputs aren't the output of a Keras layer.

Make sure that you are only passing to Model 1) inputs generated via Input 2) outputs generated by a Keras layer, with no further ops applied to them.

[HanyuGuo]

Hi,

I encounter the same problem even if I update the latest Keras ("no attribute '_keras_history'"). In my case, I concatenate a keras Input with a tensor comes from tensorflow API. I am guessing this create a problem. One example is:

concate =  keras.layers.Concatenate() 
features = []
co = tf.tile(......)
features.append(concate(F, co))   # F is Keras tensor, and co comes from Tensorflow API 
Dense(128)(features)    # features then send back to a keras layer

[leexa90]

Hi , i have the same problem too , where the final keras Model throws out the same problem. I cant seem to find a workaround and i believe the error was from K.tf.matmul which is an essential operation i need to make.

x = Input(shape=(100,5, 32)) 
final = {}
for i in range(0,10): # get 100*100 matrix for each layer with outer product
    mat_x = x[:,:,:,i]
    final[i] =   K.tf.matmul( mat_x, mat_x, transpose_b=True ) #gives 100*100  matrix

reshape_100_100_1 = Reshape((100,100,1))
y = merge([reshape_100_100_1 (final[x]) for x in final],mode='concat', concat_axis=3)
model = Model (seq_input,y)

I have tried the Lambda layer method method. using the K.dot to do matrix multiplication and they all unfortunately failed at the same step. K.dot method : https://github.com/fchollet/keras/issues/4675

Outputs of y <tf.Tensor 'reshape_3/Reshape:0' shape=(?, 100, 100, 10) dtype=float32> seq_input <tf.Tensor 'input_3:0' shape=(?, 100, 5, 32) dtype=float32>

[JianboTang]

I encounter this problem, too

[AndreJFBico]

This still happens, and using lambda expressions all over the place to fix this issue feels that it isn't correct.

[dezmoanded]

Did you try wrapping the tf operation in a Layer subclass? Maybe that's the 'correct' way to do it.

[legentz]

@dezmoanded Can you provide an example, please? Thanks :)

[dezmoanded]

For instance, if I want to use tf.image.resize_images:

class UpSamplingBilinear(Layer):
    def __init__(self, scale=4):
        self.scale = scale
        super(UpSamplingBilinear, self).__init__()

    def build(self, input_shape):
        self.shape = input_shape
        super(UpSamplingBilinear, self).build(input_shape)

    def call(self, inputs, **kwargs):
        new_size = self.compute_output_shape(self.shape)[-2:]
        x = K.permute_dimensions(inputs, [0, 2, 3, 1])
        x = tf.image.resize_images(x, new_size)
        x = K.permute_dimensions(x, [0, 3, 1, 2])
        return x

    def compute_output_shape(self, input_shape):
        input_shape = list(input_shape)
        input_shape[2] *= self.scale
        input_shape[3] *= self.scale
        return tuple(input_shape)

[legentz]

@dezmoanded Thanks but let me explain my situation. I'm not so good with Keras/Tensorflow... a n00b :)

I created a Tensor 'var', this way np_var = np.random.uniform(-0.01, 0.01, size=shape) var = constant(np_var, shape=shape, name=name)

Then, I tried to use Keras' dot() function, as follows x = dot(rep, var) x = sigmoid(x) where 'rep' is a valid Keras Tensor (_is_kerastensor returns True) and, finally, I called 'sigmoid' over the Tensor 'x'.

The fact is that I'm not able to build my model with that 'x' as output because of the exception TypeError: Output tensors to a Model must be Keras tensors. Found Tensors("Sigmoid:0", shape=(?, 113), dtype=float32)

So I tried to feed a Dense Layer with the Tensor 'x'. Now it's a Keras Tensor, shapes are ok but... 'Tensor' object has no attribute '_keras_history'

Am I missing something?

[dezmoanded]

The problem is that dot and sigmoid are functions, not layers.

Use Dot() and Activation('sigmoid') layers.

[zaid478]

Facing same issue! I was doing Andrew Ng assignment and implementing Residual Networks and suddenly the error appears! Kindly bring up the solution if some have one ?

The code is as follow: X_input = Input(input_shape)

# Zero-Padding
X = ZeroPadding2D((3, 3))(X_input)

# Stage 1
X = Conv2D(64, (7, 7), strides = (2, 2), name = 'conv1', kernel_initializer = glorot_uniform(seed=0))(X)
X = BatchNormalization(axis = 3, name = 'bn_conv1')(X)
X = Activation('relu')(X)
X = MaxPooling2D((3, 3), strides=(2, 2))(X)

# Stage 2
X = convolutional_block(X, f = 3, filters = [64, 64, 256], stage = 2, block='a', s = 1)
X = identity_block(X, 3, [64, 64, 256], stage=2, block='b')
X = identity_block(X, 3, [64, 64, 256], stage=2, block='c')

### START CODE HERE ###

# Stage 3 (≈4 lines)
X = convolutional_block(X, f = 3, filters = [128, 128, 512], stage = 3, block='a', s = 2)
X = identity_block(X, 3, [128,128,512], stage=3, block='b')
X = identity_block(X, 3, [128,128,512], stage=3, block='c')
X = identity_block(X, 3, [128,128,512], stage=3, block='d')

# Stage 4 (≈6 lines)
X = convolutional_block(X, f = 3, filters = [256, 256, 1024], stage = 4, block='a', s = 2)
X = identity_block(X, 3, [256,256,1024], stage=4, block='b')
X = identity_block(X, 3, [256,256,1024], stage=4, block='c')
X = identity_block(X, 3, [256,256,1024], stage=4, block='d')
X = identity_block(X, 3, [256,256,1024], stage=4, block='e')
X = identity_block(X, 3, [256,256,1024], stage=4, block='f')

# Stage 5 (≈3 lines)
X =  convolutional_block(X, f = 3, filters = [512,512,2048], stage = 5, block='a', s = 2)
X =  identity_block(X, 3, [256,256,2048], stage=5, block='b')
X =  identity_block(X, 3, [256,256,2048], stage=2, block='c')

# AVGPOOL (≈1 line). Use "X = AveragePooling2D(...)(X)"
X = AveragePooling2D(pool_size=(2,2),name='avg_pool',padding='same')(X)

### END CODE HERE ###

# output layer
X = Flatten()(X)
X = Dense(classes, activation='softmax', name='fc' + str(classes), kernel_initializer = glorot_uniform(seed=0))(X)
# Create model
model = Model(inputs = X_input, outputs = X)

[courteouselk]

@zaid478 please check the discussion forums. There are couple of threads that address your problem.

[zaid478]

@courteouselk where is the discussion forum ? I have tried every possible resource that i found.

[courteouselk]

@zaid478 oh, I assumed you were doing it on coursera. Anyway. The error comes from the improperly connected tensors earlier in the logic (not in the snipped you provide). Most likely there are couple of places where you do X = X + X_shortcut, while you should do X = Add()([X, X_shortcut])

[xnnba]

@courteouselk is correct!

[freesiemens]

@zaid478 @courteouselk Thanks a lot, solve my same problem!

[acarl005]

@zaid478 Link to said discussion forum https://www.coursera.org/learn/convolutional-neural-networks/discussions/weeks/2/threads/SpfP8r9jEeeKtgo7gB_lKA

[arohamirai]

@courteouselk you are awesome

[AaronDou]

@courteouselk you rock man

[huihuibo]

thanks a lot

[sushilkv2004]

@courteouselk thanks

[rajatkoner08]

I have faced the same issue while creating a model with pre-trained Mobilenet used as a feature extractor. As input is a tensorflow tensor with datatype uint8 and shape (16,128,128,3), so I have to explicitly convert it into a Keras tensor and created model in below way : ` if input_tensor is None: img_input = Input(shape=input_shape) else: if not K.is_keras_tensor(input_tensor): img_input = Input(tensor=input_tensor, shape=input_shape) img_input = K.cast(img_input, dtype='float32') else: img_input = input_tensor
x = _conv_block(img_input, 32, alpha, strides=(2, 2)) x = _depthwise_conv_block(x, 64, alpha, depth_multiplier, block_id=1)

x = _depthwise_conv_block(x, 128, alpha, depth_multiplier,
                          strides=(2, 2), block_id=2)
x = _depthwise_conv_block(x, 128, alpha, depth_multiplier, block_id=3)

x = _depthwise_conv_block(x, 256, alpha, depth_multiplier,
                          strides=(2, 2), block_id=4)
x = _depthwise_conv_block(x, 256, alpha, depth_multiplier, block_id=5)

x = _depthwise_conv_block(x, 512, alpha, depth_multiplier,
                          strides=(2, 2), block_id=6)
x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=7)
x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=8)
x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=9)
x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=10)
x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=11)

x = _depthwise_conv_block(x, 1024, alpha, depth_multiplier,
                          strides=(2, 2), block_id=12)
x = _depthwise_conv_block(x, 1024, alpha, depth_multiplier, block_id=13)

if pooling == 'avg':
    x = GlobalAveragePooling2D()(x)
elif pooling == 'max':
    x = GlobalMaxPooling2D()(x)

'''# Ensure that the model takes into account
# any potential predecessors of `input_tensor`.
if input_tensor is not None:
    inputs = get_source_inputs(input_tensor)
else:
    inputs = img_input
'''
#TODO check input tensor is keras tensor
# Create model.
model = Model(input_tensor, x, name='mobilenet_%0.2f_%s' % (alpha, rows))

Then I get the same error : File "/home/rajat/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 1674, in build_map_of_graph layer, node_index, tensor_index = tensor._keras_history AttributeError: 'Tensor' object has no attribute '_keras_history' I already debugged this and find out that build_map_of_graph from 'x' tensor throw this error as its input(img_input ) doesn't have any _keras_history. Keras history is associated with input_tensor. can anyone please help me how can I solve this?

`

[gaufung]

you are genius

[dongmeixu]

I encounter this problem, too. Is there any way to solve it?

[leexa90]

use the lambda function on the tensorflow tensor worked for me. I cant really remember how it was done since i later erased the code and directly used tensorflow, but i hope it helps.

On Tue, Dec 12, 2017 at 4:38 PM, JavaWeb-许东梅 notifications@github.com wrote:

I encounter this problem, too. Is there any way to solve it?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/fchollet/keras/issues/7362#issuecomment-350981945, or mute the thread https://github.com/notifications/unsubscribe-auth/AQ4Jsstp9-3q_gcFv9jEoMocmMq-rw9Uks5s_juHgaJpZM4OagKu .

[pdoggi]

@courteouselk thank you!!

[AndrasEros]

My mistake was while using the Functional API I didsum() instead of importing add() from layers. I thought that I already importing a sum as a layer so I just used it. Now I realized that sum is not a Keras function. Eventually it went through until I tried to compile but that is the expected behavior.

[bnzxking]

@courteouselk thank you

[manyu90]

I am using the functional API too, and it looks like writing a Lambda layer wrapper for any custom Keras backend operation suffices. No issues after this in defining the model

[healthycola]

@courteouselk Thanks! But Why does X+X_shortcut not work?

[nolanlei]

@courteouselk zan

[ganesh-ms]

@courteouselk thank-you for the right answer.

[ghost]

Hey all,

I'm seeing this issue still with CTC operations in the tensorflow backend.

I wrapped K.ctc_batch_cost in a Lambda layer and it's not returning a Keras Tensor. For example:

# wrap ctc_batch_cost in Lambda
args = [predictions, labels, input_lengths, label_lengths]
Lambda(ctc_lambda_func, output_shape=(1,), name='ctc')(args)

Where ctc_lambda_func is the same from the image_ocr.py example:

def ctc_lambda_func(args):
    '''Wrapper around backend CTC loss'''
    y_pred, labels, input_length, label_length = args
    # dictionary for ctc_loss parameters
    return K.ctc_batch_cost(labels, y_pred, input_length, label_length)

I even tried making my own Layers subclassing both Tensorflow's and Keras' Layer class:

# tried with both of these Layer classes
from tensorflow.python.keras._impl.keras.engine.base_layer import Layer
Layer = tf.layers.Layer
class CTCLayer(tf.layers.Layer):
    '''
    Custom layer to return CTC as Keras Tensor.
    '''
    def __init__(self, output_dim=1, **kwargs):
        self.output_dim = output_dim
        super(CTCLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        # build layer
        super(CTCLayer, self).build(input_shape)  # Be sure to call this somewhere!

    def call(self, args):
        y_pred, labels, input_length, label_length = args
        return K.ctc_batch_cost(labels, y_pred, input_length, label_length)

    def compute_output_shape(self, input_shape):
        return (input_shape[0], 1)

No matter what I do, anything involving K.ctc_* always returns a regular Tensor and not a Keras Tensor. This means it can't be used as the output for a model.

Has anyone dealt with this specifically for sequence/CTC problems? There really is no way to avoid ctc_batch_cost...

Thank you!

P.S. In the backend code, it seem to be this line:

    return tf.expand_dims(ctc.ctc_loss(inputs=y_pred,
                                       labels=sparse_labels,
                                       sequence_length=input_length), 1)

That expand_dims is never cast to a Keras Tensor, and is forever just Tensor.

[krajkumar6]

@courteouselk , Thanks a ton!! I did just that and it worked fine

[Zhangtd]

I encountered this problem when tried to slice an Input layer, just like i1 = Input(shape=(batch_size, 2, 3, 3)) d1 = Dense(3)(i1[:, 1, :, :])

And it was solved after I modified this operation.

[FelixOtt94]

Hi Zhangtd,

I tried to slice a Tensor the same way you wanted to and got the error: 'Tensor' object has no attribute '_keras_history' I get from my network a Tensor of shape=(?,3,32,32) and want to put it into three separate LSTMs with Input shape=(?,1,32,32). Therefore, I sliced the Tensor and faced the same problem. So how did you modified your operation to solve the problem? This would help me a lot.

Thanks in advance!

[Zhangtd]

I just split the input Tensor into 3 input tensors. That is, from one shape=(?, 3, 32, 32) to three shape=(?, 32, 32). It is silly but solved the problem. Please inform me of the method if you solve the problem in a more elegant way.

---

发件人: FelixOtt94 notifications@github.com 发送时间: 2018年7月5日 16:23 收件人: keras-team/keras 抄送: Suwei Zhang; Comment 主题: Re: [keras-team/keras] Creating Model variable throws "AttributeError: 'Tensor' object has no attribute '_keras_history'" (#7362)

Hi Zhangtd,

I tried to slice a Tensor the same way you wanted to and got the error: 'Tensor' object has no attribute '_keras_history' I get from my network a Tensor of shape=(?,3,32,32) and want to put it into three separate LSTMs with Input shape=(?,1,32,32). Therefore, I sliced the Tensor and faced the same problem. So how did you modified your operation to solve the problem? This would help me a lot.

Thanks in advance!

― You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/keras-team/keras/issues/7362#issuecomment-402644120, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AT6hLlQ3CHvKjTa5RHxJ8zqzaAULZ4cVks5uDczwgaJpZM4OagKu.

[richardrl]

@courteouselk gave the right solution, but in general, how do you diagnose an error like this in Keras? The error given was completely nondescriptive as to what line # or true cause of the error was...

[bibin-sebastian]

I agree with @richardrl. Is there a way to quickly get to the crust of the issue rather than not understanding anything from the Attribute error verbose

[BassantTolba]

Please all, I have this error and i spent lot of time to solve this . "Tensor has no attribute outputs" ,Below there is the error figure and the my code . Please i need a solution to correct and solve this. The error appears in this line

_, logits_fake = SRGAN_d(net_g.outputs, is_train=True, reuse=True)

-- coding: utf8 --

import os, time, pickle, random, time from datetime import datetime import numpy as np from time import localtime, strftime import logging import scipy.io as sio

import tensorflow as tf import tensorlayer as tl from model import SRGAN_g, SRGAN_d from config import config, log_config envir = 'indoor' #'indoor' or 'outdoor'

image parameter

img_height = 32 img_width = 32 img_channels = 2 img_total = img_heightimg_widthimg_channels

network params

residual_num = 2 encoded_dim = 512 #compress rate=1/4->dim.=512, compress rate=1/16->dim.=128, compress rate=1/32->dim.=64, compress rate=1/64->dim.=32

====================== HYPER-PARAMETERS ===========================

Adam

batch_size = config.TRAIN.batch_size lr_init = config.TRAIN.lr_init beta1 = config.TRAIN.beta1

initialize G

n_epoch_init = config.TRAIN.n_epoch_init

adversarial learning (SRGAN)

n_epoch = config.TRAIN.n_epoch lr_decay = config.TRAIN.lr_decay decay_every = config.TRAIN.decay_every

ni = int(np.sqrt(batch_size))

def train():

create folders to save result images and trained model

save_dir_ginit = "samples/{}_ginit".format(tl.global_flag['mode'])
save_dir_gan = "samples/{}_gan".format(tl.global_flag['mode'])
tl.files.exists_or_mkdir(save_dir_ginit)
tl.files.exists_or_mkdir(save_dir_gan)
checkpoint_dir = "checkpoint"  # checkpoint_resize_conv
tl.files.exists_or_mkdir(checkpoint_dir)

###====================== PRE-LOAD DATA ===========================###
if envir == 'indoor':
    mat = sio.loadmat('data/DATA_Htrainin.mat')
    x_train = mat['HT'] # array
    mat = sio.loadmat('data/DATA_Hvalin.mat')
    x_val = mat['HT'] # array
    mat = sio.loadmat('data/DATA_Htestin.mat')
    x_test = mat['HT'] # array
elif envir == 'outdoor':
    mat = sio.loadmat('data/DATA_Htrainout.mat')
    x_train = mat['HT'] # array
    mat = sio.loadmat('data/DATA_Hvalout.mat')
    x_val = mat['HT'] # array
    mat = sio.loadmat('data/DATA_Htestout.mat')
    x_test = mat['HT'] # array

x_train = x_train.astype('float32')
x_val = x_val.astype('float32')
x_test = x_test.astype('float32')
x_train = np.reshape(x_train, (len(x_train), img_channels, img_height, img_width))  # adapt this if using `channels_first` image data format
x_val = np.reshape(x_val, (len(x_val), img_channels, img_height, img_width))  # adapt this if using `channels_first` image data format
x_test = np.reshape(x_test, (len(x_test), img_channels, img_height, img_width))  # adapt this if using `channels_first` image data format

###========================== DEFINE MODEL ============================###
## train inference
image_tensor = tf.placeholder('float32', [batch_size, img_channels, img_height, img_width], name='t_image_input_to_SRGAN_generator')
t_target_image = tf.placeholder('float32', [batch_size, img_channels, img_height, img_width], name='t_target_image')

net_g = SRGAN_g(image_tensor, residual_num, encoded_dim)
net_d, logits_real = SRGAN_d(t_target_image, is_train=True, reuse=False)
_, logits_fake = SRGAN_d(net_g.outputs, is_train=True, reuse=True)

net_g.print_params(False)
net_g.print_layers()
net_d.print_params(False)
net_d.print_layers()

[mrgloom]

@dezmoanded I have used your example of UpSamplingBilinear, trying to wrap tf function with keras layer but still get AttributeError: 'Tensor' object has no attribute '_keras_history'

#(None,6)->(None,8)
class AffineToProjectiveZeroPadding1D(Layer):
    def __init__(self):
        super(AffineToProjectiveZeroPadding1D, self).__init__()

    def build(self, input_shape):
        self.shape = input_shape
        super(AffineToProjectiveZeroPadding1D, self).build(input_shape)

    def call(self, x):
        paddings = tf.constant([[0, 0], [0, 2]])
        projective_transform = tf.pad(x,
                                      paddings,
                                      mode='CONSTANT',
                                      constant_values=0
                                      )
        return projective_transform

    def compute_output_shape(self, input_shape):
        output_shape = (input_shape[0], input_shape[1]+2)
        return output_shape

Looks like it easy with Lamda layer:

    # (None, 6) -> (None, 8)
    def affine_to_projective(x):
        y = tf.pad(x,
                   paddings=tf.constant([[0, 0], [0, 2]]),
                   mode='CONSTANT',
                   constant_values=0
                   )
        return y

    projective_transform = Lambda(affine_to_projective)(affine_transform)

[snknitin]

@courteouselk Thanks. That saved me a lot of trouble. I wasn't using Add()

[shivendrapratap2]

@zaid478 oh, I assumed you were doing it on coursera. Anyway. The error comes from the improperly connected tensors earlier in the logic (not in the snipped you provide). Most likely there are couple of places where you do X = X + X_shortcut, while you should do X = Add()([X, X_shortcut])

i owe you a party bro : P

[baabaablacksheep]

Thanks a lot. I was doing a course on Coursera and Stucked for hours in this error. Thanks

[HGamalElDin]

please @courteouselk I'm faceing same error

AttributeError: in user code:

    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py:533 train_step  **
        y, y_pred, sample_weight, regularization_losses=self.losses)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\engine\compile_utils.py:205 __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\losses.py:143 __call__
        losses = self.call(y_true, y_pred)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\losses.py:246 call
        return self.fn(y_true, y_pred, **self._fn_kwargs)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\keras_contrib\losses\crf_losses.py:54 crf_loss
        crf, idx = y_pred._keras_history[:2]
AttributeError: 'Tensor' object has no attribute '_keras_history'

without concatenation function at all Model Architecture:

from keras.models import Model, Input
from tensorflow.keras.layers import LSTM, Embedding, Dense, TimeDistributed, Bidirectional
import tensorflow.keras as k
from keras_contrib.layers import CRF

input = Input(shape=(maxlen,))
word_embedding_size = 150

MODEL = Embedding(input_dim=n_words, output_dim=word_embedding_size, input_length=140)(input)

MODEL = Bidirectional(LSTM(units=word_embedding_size, 
                           return_sequences=True, 
                           dropout=0.5, 
                           recurrent_dropout=0.5, 
                           kernel_initializer=k.initializers.he_normal()))(MODEL)
MODEL = LSTM(units=word_embedding_size * 2, 
             return_sequences=True, 
             dropout=0.5, 
             recurrent_dropout=0.5, 
             kernel_initializer=k.initializers.he_normal())(MODEL)

MODEL = TimeDistributed(Dense(n_tags, activation="relu"))(MODEL)  

CRF_LAYER = CRF(n_tags)

OUTPUT = CRF_LAYER(MODEL)  # OUTPUT
MODEL = Model(input, OUTPUT)

Spec: Keras=2.4.3 python=3.7 Windows

[poppybrown]

I refer to this link https://blog.csdn.net/qq_45947969/article/details/109392826，You should ensure that the keras and tf versions correspond

[KINJALMARU16]

@zaid478 oh, I assumed you were doing it on coursera. Anyway. The error comes from the improperly connected tensors earlier in the logic (not in the snipped you provide). Most likely there are couple of places where you do X = X + X_shortcut, while you should do X = Add()([X, X_shortcut])

Hi , I did not get this. I am new to Bi-lstm CRF training. But I am getting the same error

[dongmeixu]

您好，已收到您的来信，谢谢~~~

[KINJALMARU16]

please @courteouselk I'm faceing same error

AttributeError: in user code:

    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py:533 train_step  **
        y, y_pred, sample_weight, regularization_losses=self.losses)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\engine\compile_utils.py:205 __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\losses.py:143 __call__
        losses = self.call(y_true, y_pred)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\tensorflow\python\keras\losses.py:246 call
        return self.fn(y_true, y_pred, **self._fn_kwargs)
    C:\Users\Heba Gamal El-Din\anaconda3\lib\site-packages\keras_contrib\losses\crf_losses.py:54 crf_loss
        crf, idx = y_pred._keras_history[:2]
AttributeError: 'Tensor' object has no attribute '_keras_history'

without concatenation function at all Model Architecture:

from keras.models import Model, Input
from tensorflow.keras.layers import LSTM, Embedding, Dense, TimeDistributed, Bidirectional
import tensorflow.keras as k
from keras_contrib.layers import CRF

input = Input(shape=(maxlen,))
word_embedding_size = 150

MODEL = Embedding(input_dim=n_words, output_dim=word_embedding_size, input_length=140)(input)

MODEL = Bidirectional(LSTM(units=word_embedding_size, 
                           return_sequences=True, 
                           dropout=0.5, 
                           recurrent_dropout=0.5, 
                           kernel_initializer=k.initializers.he_normal()))(MODEL)
MODEL = LSTM(units=word_embedding_size * 2, 
             return_sequences=True, 
             dropout=0.5, 
             recurrent_dropout=0.5, 
             kernel_initializer=k.initializers.he_normal())(MODEL)

MODEL = TimeDistributed(Dense(n_tags, activation="relu"))(MODEL)  

CRF_LAYER = CRF(n_tags)

OUTPUT = CRF_LAYER(MODEL)  # OUTPUT
MODEL = Model(input, OUTPUT)

Spec: Keras=2.4.3 python=3.7 Windows

Hi @HGamalElDin Were you able to resolve the issue?