model with dropout layer has shape mismatch problem during batch prediction

[Kuzzzz]

When i use tensorflow/serving docker to deploy my tf model with dropout layers, i set --enable_batching=true to ensure batch prediction can work. But when i use Restful API service for prediction and request url with data like: {"instances": [{xxxxx, "dropout":1.0}]} (len(batch)==1 can recieve correct result), if i use data like: {"instances": [{xxxxx, "dropout":1.0}， {xxxxx, "dropout":1.0}， {xxxxx, "dropout":1.0}]} (len(batch) > 1) I will recieve 【{'error': 'Incompatible shapes: [3,300,120] vs. [3]\n\t [[{{node dropout_1/div}}]]'}】 Why did i encounter this situation?

[rmothukuru]

@Kuzzzz, Can you please share the SignatureDef of your Model. Thanks!

[rmothukuru]

@Kuzzzz , Can you please respond to the comment above. Thanks!

[Kuzzzz]

@rmothukuru ,sorry i forgot to watch github recently, here is my model`s signatureDef: signature_def['serving_default']: The given SavedModel SignatureDef contains the following input(s): inputs['dropout_keep_prob'] tensor_info: dtype: DT_FLOAT shape: unknown_rank name: dropout_keep_prob:0 inputs['input_p1'] tensor_info: dtype: DT_INT32 shape: (-1, -1) name: input_p1:0 inputs['input_p2'] tensor_info: dtype: DT_INT32 shape: (-1, -1) name: input_p2:0 inputs['input_x'] tensor_info: dtype: DT_FLOAT shape: (-1, -1, 100) name: input_x:0 The given SavedModel SignatureDef contains the following output(s): outputs['prediction'] tensor_info: dtype: DT_INT64 shape: (-1) name: output/predictions:0 Method name is: tensorflow/serving/predict

[Kuzzzz]

another question i met is：when i set --enable_batching=False, my request can be predicted correctly. But when i set --enable_batching=True, dimensions error will appear. may be when i dont use batching, shape of my input tensors are input_x:(1, 18, 100) input_p1:(1, 18, 5) input_p2:(1, 18, 5), but when i use batching, they change to (1, 1, 18, 100) (1, 1, 18, 5) (1, 1, 18, 5) . if i use tf.expand_dims() in my model, something will be wrong because of "dimensions error"

[Kuzzzz]

another question i met is：when i set --enable_batching=False, my request can be predicted correctly. But when i set --enable_batching=True, dimensions error will appear. may be when i dont use batching, shape of my input tensors are input_x:(1, 18, 100) input_p1:(1, 18, 5) input_p2:(1, 18, 5), but when i use batching, they change to (1, 1, 18, 100) (1, 1, 18, 5) (1, 1, 18, 5) . if i use tf.expand_dims() in my model, something will be wrong because of "dimensions error"

this question is caused by my data error, only dropout layer dimention error is still unsolved.

[gowthamkpr]

@Kuzzzz please provide the code and exact steps to reproduce this issue. And also this error is caused by tensorflow itself not serving.Thanks!

[gowthamkpr]

@Kuzzzz Can you please respond to the above comment. Thanks!

[gowthamkpr]

Closing this issue as it has been inactive for more than 2 weeks. Please add additional comments for us to open this issue again. Thanks!