Closed kt1004 closed 6 years ago
Have you found solution? Having exactly same problem!
Figured it out. Make a regress
call instead of predict
curl -d '{"examples": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}'
-X POST http://localhost:8501/v1/models/census:regress
After serving the model using tensorflow serving. I used the same instances as that if your, didn't work for me
Got this error:
curl -d '{"examples": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}' -X POST http://localhost:8501/v1/models/census:regress
{ "error": "Expected regression signature method_name to be tensorflow/serving/regress. Was: tensorflow/serving/classify" }
Instead of regress i used classify, then having this error:
curl -d '{"examples": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}' -X POST http://localhost:8501/v1/models/census:classify
{ "error": "Name: <unknown>, Key: hours-per-week, Index: 0. Data types don\'t match. Data type: int64 but expected type: float\n\t [[{{node ParseExample/ParseExample}}]]" }
The index parameter at 0 keeps changing with different parameter names.
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['classification']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 2)
name: linear/head/Tile:0
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 2)
name: linear/head/predictions/probabilities:0
Method name is: tensorflow/serving/classify
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['examples'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['all_class_ids'] tensor_info:
dtype: DT_INT32
shape: (-1, 2)
name: linear/head/predictions/Tile:0
outputs['all_classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 2)
name: linear/head/predictions/Tile_1:0
outputs['class_ids'] tensor_info:
dtype: DT_INT64
shape: (-1, 1)
name: linear/head/predictions/ExpandDims:0
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 1)
name: linear/head/predictions/str_classes:0
outputs['logistic'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 1)
name: linear/head/predictions/logistic:0
outputs['logits'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 1)
name: linear/linear_model/linear_model/linear_model/weighted_sum:0
outputs['probabilities'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 2)
name: linear/head/predictions/probabilities:0
Method name is: tensorflow/serving/predict
signature_def['regression']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['outputs'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 1)
name: linear/head/predictions/logistic:0
Method name is: tensorflow/serving/regress
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 2)
name: linear/head/Tile:0
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 2)
name: linear/head/predictions/probabilities:0
Method name is: tensorflow/serving/classify
It was a bit random, however, this curl was able to solve.
~$ curl -d '{"examples": [{"age":50.0, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13.0, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0.0, "capital-loss":0.0, "hours-per-week":13.0, "native-country":"United-States"}]}' -X POST http://localhost:8501/v1/models/census:classify
{
"results": [[["0", 0.426006019], ["1", 0.573994]]
]
Figured it out. Make a
regress
call instead ofpredict
curl -d '{"examples": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}' -X POST http://localhost:8501/v1/models/census:regress
How do you manage to call regress
? I got { "error": "Expected regression signature method_name to be tensorflow/serving/regress. Was: tensorflow/serving/predict" }
when I tried to call this method. Please help
Check this out: https://medium.com/delvify/bert-rest-inference-from-the-fine-tuned-model-499997b32851
Hi, thank you for the link.
Like the example in this article, I used a dict of tf.placeholder directly as the receiver tensor and the predict
can be successfully called.
I also checked the available method from metadata
endpoint and I noticed that predict
is the only one available for served. I suspect that the model exported from tf.estimator only has predict
as the available method in its signature def
I run census_example.py and I check exported model file. https://github.com/tensorflow/transform/blob/master/examples/census_example.py
And, I can check ServingInputReceiver below print("serving_input_receiver:", serving_input_receiver) ==> ServingInputReceiver(features={ 'capital-loss': <tf.Tensor 'ParseExample/ParseExample:2' shape=(?,) dtype=float32>, 'relationship': <tf.Tensor 'ParseExample/ParseExample:10' shape=(?,) dtype=string>, 'age': <tf.Tensor 'ParseExample/ParseExample:0' shape=(?,) dtype=float32>, ... receiver_tensors={'examples': <tf.Tensor 'input_example_tensor:0' shape=(?,) dtype=string>}, receiver_tensors_alternatives=None)
And I run tensorflow serving using the model in exported model dir. And I send curl request but, I got error
curl -d '{"instances": [{"age":50, "workclass":"Self-emp-not-inc", "education":"Bachelors", "education-num":13, "marital-status":"Married-civ-spouse", "occupation":"Exec-managerial", "relationship":"Husband", "race":"White", "sex":"Male", "capital-gain":0, "capital-loss":0, "hours-per-week":13, "native-country":"United-States"}]}' -X POST http://localhost:8501/v1/models/census:predict
{ "error": "Failed to process element: 0 key: age of \'instances\' list. Error: Invalid argument: JSON object: does not have named input: age" }
How do I call ?