tf transform's "exported model" use in tf serving issue

[kt1004]

I run census_example.py and I check exported model file. https://github.com/tensorflow/transform/blob/master/examples/census_example.py

And, I can check ServingInputReceiver below print("serving_input_receiver:", serving_input_receiver) ==> ServingInputReceiver(features={ 'capital-loss': <tf.Tensor 'ParseExample/ParseExample:2' shape=(?,) dtype=float32>, 'relationship': <tf.Tensor 'ParseExample/ParseExample:10' shape=(?,) dtype=string>, 'age': <tf.Tensor 'ParseExample/ParseExample:0' shape=(?,) dtype=float32>, ... receiver_tensors={'examples': <tf.Tensor 'input_example_tensor:0' shape=(?,) dtype=string>}, receiver_tensors_alternatives=None)

And I run tensorflow serving using the model in exported model dir. And I send curl request but, I got error

curl -d '{"instances": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}' -X POST http://localhost:8501/v1/models/census:predict

{ "error": "Failed to process element: 0 key: age of \'instances\' list. Error: Invalid argument: JSON object: does not have named input: age" }

How do I call ?

[sergpsu]

Have you found solution? Having exactly same problem!

[jlertle]

Figured it out. Make a regress call instead of predict

curl -d '{"examples": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}'
-X POST http://localhost:8501/v1/models/census:regress

Docs: https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/api_rest.md#make-rest-api-calls-to-modelserver

[jageshmaharjan]

After serving the model using tensorflow serving. I used the same instances as that if your, didn't work for me

Got this error:

curl -d '{"examples": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}' -X POST http://localhost:8501/v1/models/census:regress
{ "error": "Expected regression signature method_name to be tensorflow/serving/regress. Was: tensorflow/serving/classify" }

Instead of regress i used classify, then having this error:

 curl -d '{"examples": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}' -X POST http://localhost:8501/v1/models/census:classify
{ "error": "Name: <unknown>, Key: hours-per-week, Index: 0.  Data types don\'t match. Data type: int64 but expected type: float\n\t [[{{node ParseExample/ParseExample}}]]" }

The index parameter at 0 keeps changing with different parameter names.

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['classification']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['inputs'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: input_example_tensor:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['classes'] tensor_info:
        dtype: DT_STRING
        shape: (-1, 2)
        name: linear/head/Tile:0
    outputs['scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 2)
        name: linear/head/predictions/probabilities:0
  Method name is: tensorflow/serving/classify

signature_def['predict']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['examples'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: input_example_tensor:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['all_class_ids'] tensor_info:
        dtype: DT_INT32
        shape: (-1, 2)
        name: linear/head/predictions/Tile:0
    outputs['all_classes'] tensor_info:
        dtype: DT_STRING
        shape: (-1, 2)
        name: linear/head/predictions/Tile_1:0
    outputs['class_ids'] tensor_info:
        dtype: DT_INT64
        shape: (-1, 1)
        name: linear/head/predictions/ExpandDims:0
    outputs['classes'] tensor_info:
        dtype: DT_STRING
        shape: (-1, 1)
        name: linear/head/predictions/str_classes:0
    outputs['logistic'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: linear/head/predictions/logistic:0
    outputs['logits'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: linear/linear_model/linear_model/linear_model/weighted_sum:0
    outputs['probabilities'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 2)
        name: linear/head/predictions/probabilities:0
  Method name is: tensorflow/serving/predict

signature_def['regression']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['inputs'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: input_example_tensor:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['outputs'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: linear/head/predictions/logistic:0
  Method name is: tensorflow/serving/regress

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['inputs'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: input_example_tensor:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['classes'] tensor_info:
        dtype: DT_STRING
        shape: (-1, 2)
        name: linear/head/Tile:0
    outputs['scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 2)
        name: linear/head/predictions/probabilities:0
  Method name is: tensorflow/serving/classify

[jageshmaharjan]

It was a bit random, however, this curl was able to solve.

~$ curl -d '{"examples": [{"age":50.0, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13.0, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0.0, "capital-loss":0.0, "hours-per-week":13.0, "native-country":"United-States"}]}' -X POST http://localhost:8501/v1/models/census:classify
{
    "results": [[["0", 0.426006019], ["1", 0.573994]]
    ]

[suisenkotoba]

Figured it out. Make a regress call instead of predict

curl -d '{"examples": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}'
-X POST http://localhost:8501/v1/models/census:regress

Docs: https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/api_rest.md#make-rest-api-calls-to-modelserver

How do you manage to call regress ? I got { "error": "Expected regression signature method_name to be tensorflow/serving/regress. Was: tensorflow/serving/predict" } when I tried to call this method. Please help

[jageshmaharjan]

Check this out: https://medium.com/delvify/bert-rest-inference-from-the-fine-tuned-model-499997b32851

[suisenkotoba]

Check this out: https://medium.com/delvify/bert-rest-inference-from-the-fine-tuned-model-499997b32851

Hi, thank you for the link. Like the example in this article, I used a dict of tf.placeholder directly as the receiver tensor and the predict can be successfully called. I also checked the available method from metadata endpoint and I noticed that predict is the only one available for served. I suspect that the model exported from tf.estimator only has predict as the available method in its signature def