Closed azagniotov closed 4 years ago
@azagniotov The serving_input_fn
expects a serialized ELWC
proto, not the raw Tensor values. Can you try serializing the proto?
Hi @ramakumar1729 , thank you.
Just to make sure I got this right: you are saying that when I make the curl request, the request payload should be a serialized proto instead of the raw {"ctx_f0": 7.2} ...
?
@ramakumar1729 on a related topic:
I looked into a different way of generating the serving_input_receiver_fn
, which allowed me to POST the aforementioned JSON payload to the server and obtain model results without any errors. The following is the code that I used to build the serving_input_receiver_fn
:
def all_feature_spec():
feature_names = ['f0', 'f1', 'f2']
features = {
name: tf.compat.v1.FixedLenFeature([1], tf.float32) for name in feature_names
}
features['ctx_f0'] = tf.compat.v1.FixedLenFeature([1], tf.int64)
return features
serving_input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(all_feature_spec())
Could you comment (pros/cons/edge cases) on using the API tf.estimator.export.build_parsing_serving_input_receiver_fn(..)
VS using the tfr.data.build_ranking_serving_input_receiver_fn(...)
to build the serving_input_receiver_fn
? It appears that the former does enable me to POST the following JSON with raw numeric values to the REST API without any errors:
{"context": {"ctx_f0": 7.2}, "examples":[{"f0":[35.92],"f1":[5.258],"f2":[5.261]},{"f0":[82.337],"f1":[2.06],"f2":[2.068]}]}
@azagniotov : TF Serving supports few different kinds of SignatureDefs, for classify
, regress
and the generic predict
apis.
In your cURL request, you are using a regress API call, which only supports tf.Examples as inputs (you are passing features for each example separately). To generate scores for all examples in a list using ELWC format, you need to use the predict
api. I guess regress
API allows for raw numeric values but I'm not sure predict
API does.
Let me know if you have any further questions, happy to help.
@ramakumar1729 thank you for your patience.
I have created a serialized proto:
import random
from random import randint
def _float_feature(value):
return tf.train.Feature(float_list=tf.train.FloatList(value=[value]))
def create_example():
feature_names = ['f0', 'f1', 'f2']
features = {name: _float_feature(random.uniform(300, 10)) for name in feature_names}
return tf.train.Example(features=tf.train.Features(feature=features))
def create_serialized_example_list(num_of_examples):
example_protos = []
for x in range(num_of_examples):
example_proto = create_example()
print(example_proto)
example_proto_serialized = example_proto.SerializeToString()
print(example_proto_serialized)
example_protos.append(example_proto_serialized)
return tf.make_tensor_proto(example_protos, dtype=tf.string)
def create_serialized_context():
feature_names = ['ctx_f0']
features = {name: _float_feature(random.uniform(300, 10)) for name in feature_names}
context_example = tf.train.Example(features=tf.train.Features(feature=features))
context_example_proto = context_example.SerializeToString()
return tf.make_tensor_proto(context_example_proto, dtype=tf.string)
examples_list = create_serialized_example_list(2)
print(examples_list)
context = create_serialized_context()
print(context)
For visibility & debugging purposes, the print(examples_list)
converts the following example proto:
features {
feature {
key: "f0"
value {
float_list {
value: 238.68080139160156
}
}
}
feature {
key: "f1"
value {
float_list {
value: 297.0102233886719
}
}
}
feature {
key: "f2"
value {
float_list {
value: 22.836402893066406
}
}
}
}
...
...
features { ... }
to the serialized proto blob:
dtype: DT_STRING
tensor_shape {
dim {
size: 2
}
}
string_val: "\n0\n\016\n\002f0\022\010\022\006\n\004S\003\201C\n\016\n\002f1\022\010\022\006\n\004z\300CC\n\016\n\002f2\022\010\022\006\n\004\243\307\tC"
string_val: "\n0\n\016\n\002f2\022\010\022\006\n\004\266\271oC\n\016\n\002f0\022\010\022\006\n\004\350E\204B\n\016\n\002f1\022\010\022\006\n\004\214\352\203C"
while the print(context)
produces the:
dtype: DT_STRING
tensor_shape {
}
string_val: "\n\024\n\022\n\006ctx_f0\022\010\022\006\n\004\307S\234B"
Now that I have the examples in the list & the context feature protos serialized, can you advise what should the cURL request structure look like, what should be the -d
param payload in:
curl -H "Content-Type: application/json" \
-X POST http://192.168.99.100:8501/v1/models/my_model/versions/1587842143:<METHOD> \
-d '?????'
You can take a look at the resnet client example to create PredictRequests. Based on your receiver_name
, the -d
param payload should be something like "{'input_ranking_data': <serialized_elwc>}"
. We usually do this programmatically, similar to the resnet client.
@ramakumar1729, thank you.
I am still not fully clear about the structure of serialized_elwc
. Based on the debug output generated by my functions above, I have two examples and one context feature as serialized protos. If I read this correctly and by looking at ResNet client example, the payload should be:
{"input_ranking_data": [serialized_example, serialized_example, serialized_context]
i.e.:
{"input_ranking_data": [
"\n0\n\016\n\002f0\022\010\022\006\n\004S\003\201C\n\016\n\002f1\022\010\022\006\n\004z\300CC\n\016\n\002f2\022\010\022\006\n\004\243\307\tC",
"\n0\n\016\n\002f2\022\010\022\006\n\004\266\271oC\n\016\n\002f0\022\010\022\006\n\004\350E\204B\n\016\n\002f1\022\010\022\006\n\004\214\352\203C",
"\n\024\n\022\n\006ctx_f0\022\010\022\006\n\004\307S\234B"]}
Please let me know if I blatantly misunderstood you
EDIT: That would not work as JSON parse won't parse that ^. I will try encoding it as base64 as per ResNet
Also @ramakumar1729, another interesting fact: despite what the receiver_name
may be set to, regress
& classify
methods require examples
word to be passed as a receiver_name
, while predict
method name requires instances
or inputs
:
curl -H "Content-Type: application/json" \
> -X POST http://192.168.99.100:8501/v1/models/tf_ranking_v8_local/versions/1588000040:predict \
> -d '{"input_ranking_data":[]}'
{ "error": "Missing \'inputs\' or \'instances\' key" }
curl -H "Content-Type: application/json" \
> -X POST http://192.168.99.100:8501/v1/models/tf_ranking_v8_local/versions/1588000040:classify \
> -d '{"input_ranking_data":[]}'
{ "error": "JSON Value: {\n \"input_ranking_data\": []\n} When method is classify, key \'examples\' is expected and was not found" }
curl -H "Content-Type: application/json" \
> -X POST http://192.168.99.100:8501/v1/models/tf_ranking_v8_local/versions/1588000040:regress \
> -d '{"input_ranking_data":[]}'
{ "error": "JSON Value: {\n \"input_ranking_data\": []\n} When method is classify, key \'examples\' is expected and was not found" }
@ramakumar1729, I got the making request to the REST API to work when hitting predict
API:
import random
import base64
from random import randint
def _float_feature(value):
return tf.train.Feature(float_list=tf.train.FloatList(value=[value]))
def _int_feature(value):
return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))
def create_example():
feature_names = ['f0', 'f1', 'f2']
features = {name: _float_feature(random.uniform(300, 10)) for name in feature_names}
features['ctx_f0'] = _int_feature(123)
return tf.train.Example(features=tf.train.Features(feature=features))
def create_serialized_example_list(num_of_examples):
example_protos = []
for x in range(num_of_examples):
example_proto = create_example()
example_proto_serialized = example_proto.SerializeToString()
example_protos.append(example_proto_serialized)
return tf.make_tensor_proto(example_protos, dtype=tf.string)
tensor_proto = create_serialized_example_list(20)
tensor_proto_serialized = tensor_proto.SerializeToString()
tensor_proto_base64_bytes = base64.b64encode(tensor_proto_serialized)
tensor_proto_base64 = tensor_proto_base64_bytes.decode('utf-8')
print(tensor_proto_base64)
and the cURL call is:
the tensor_proto_base64
holds the CAcSBBICCBRCQwpBCg4K.....
string
curl -H "Content-Type: application/json" \
-X POST http://192.168.99.100:8501/v1/models/tf_ranking_v9/versions/1588000040:predict \
-d '{"instances": [{"b64": "CAcSBBICCBRCQwpBCg4K..... TRUNCATED"}]}'
The results are as follows:
{
"predictions": [[0.0222161338, 0.0222163089, 0.02221632, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162828, 0.0222162679, 0.0222161338]
]
}
What's interesting though is that I always get this ^ same result set, no matter what the input is (i.e.: what the feature numeric values are). I can make 3 different cURL requests with a different b64
string as a payload, but the predictions will always be as the aforementioned ^
In fact, I can use completely bogus example feature names (i.e.: apples
, oranges
& lemons
) when constructing the tensor_proto, the API still is able to return the same above predictions. Should not the model complain that the expected features f0
, f1
and f2
are not there?
Any thoughts?
The predict serving API does not work well with cURL request afaik. If you want to make a cURL request, please use the regress serving api:
curl -d '{"signature_name": "regression", "examples": [{"f_0": [val], "f_1": [1681]}, {"f_0": 1100, "f_1": 2000}]}' \
-X POST http://192.168.99.100:8501/v1/models/tf_ranking_v9/versions/1588000040
Hi @ramakumar1729 ,
Yes, using regress
REST API has been working for me already, e.g.:
curl -H "Content-Type: application/json" \
-X POST http://192.168.99.100:8501/v1/models/tf_ranking_v8_local/versions/1587797526:regress \
-d '{"context": {"ctx_f0": 7}, "examples":[{"f1":[335.92166],"f2":[5.2585354E-9],"f3":[5.26169E-4]},{"f1":[82.33758],"f2":[2.068338E-7],"f3":[2.0684237E-7]}]}'
The original reason for the current issue was to understand the gotchas around making inference requests after generating serving_input_receiver_fn
using tfr.data.build_ranking_serving_input_receiver_fn
, which I clarified with your pointers, thank you.
I do have a few more questions:
From what I see it is possible to use predict
API when making a request with raw numeric values JSON to the saved_model_cli
(https://github.com/tensorflow/ranking/issues/46#issuecomment-523419410). I am curious why the behavior when using saved_model_cli
and tensorflow server REST API is not consistent? i.e.: I can submit raw values to the CLI, while I need to serialize the proto and base64 encode it when POSTing to the REST API.
Why the regress
REST API allows for raw numeric values, while predict
does not? Is this by design or an overlook (i.e.: bug). If this is by design, what is the rationale behind this decision? It would be really great is the REST API contract across all REST APIs would be consistent, i.e.: would be possible to POST JSONs with raw numeric values.
If I have a listwise
trained model, can I use the regress
API for inference requests? If I can use regress
API for listwise
models, what is the input format to issue to the regress
sig that is compatible with a listwise
trained model?
Thank you
@azagniotov : TF Serving would have more details on this. For the listwise trained models provided by TF-Ranking, we support regress
and predict
API.
For regress
API, we expect tf.Examples
as input. Maybe regress
API allows for interchangeably using tf.Examples
and raw Tensors, it depends on how the POST request is processed. Note that this would only work for group_size=1
and assuming you do not have a complicated transform_fn
that works on multiple examples in an ELWC
.
For predict
API, we expect ELWC
s as input. This is something we have greater control over, and we create a placeholder
for the ELWC. Hope this helps clarify things.
@ramakumar1729 yes, thank you!
I will close the current issue.
Hi @ramakumar1729 ,
Sorry to comment on this closed issue, but I thought I would share my latest (positive & successful) findings regarding making requests to predict
serving REST API. Perhaps others may find them useful:
I was able to successfully POST serialized & base64 encoded ELWC proto to the predict
REST API and get the expected predictions. These predictions match exactly the predictions that I get if I make a gRPC request using the same ELWC proto to the TensorFlow model server over gRPC.
This gave me the confidence in behavior parity that making inference requests over HTTP vs gRPC produces consistent results for the same ELWC.
Previously in https://github.com/tensorflow/ranking/issues/189#issuecomment-620256362, I was creating a tensor_proto
out of the serialized ELWC proto, then serializing the tensor_proto & base64 that, which then I was POSTing to the API.
I found out that I should not create tensor_proto
out of serialized ELWC proto string. What needs to be serialized & base64 encoded is the ELWC proto itself. As a result, my cURL looks like as before before, the difference is that the b64
string holds the ELWC proto:
curl -H "Content-Type: application/json" -X POST \
http://192.168.99.100:8501/v1/models/tf_ranking_v10/versions/1589680164:predict \
-d '{"instances": [{"b64": "CqABCp0BCiQK.... TRUNCATED"}]}'
We can be a little more descriptive by specifying the signature_name
and the receiver_name
(whatever was defined when creating serving_input_receiver_fn
):
curl -H "Content-Type: application/json" -X POST \
http://192.168.99.100:8501/v1/models/tf_ranking_v10/versions/1589680164:predict \
-d '{"signature_name": "predict", "instances": [{"input_ranking_data": {"b64": "CqABCp0BCiQK.... TRUNCATED"}}]}'
The predictions from the above cURL requests match the gRPC response prediction after making the gRPC request as follows:
import grpc
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc
from tensorflow_serving.apis import input_pb2
from google.protobuf import text_format
from google.protobuf.json_format import MessageToDict
EXAMPLE_LIST_WITH_CONTEXT_PROTO = text_format.Parse(
"""
examples {
....
}
context {
.....
}
""", input_pb2.ExampleListWithContext())
example_list_with_context_proto = EXAMPLE_LIST_WITH_CONTEXT_PROTO.SerializeToString()
tensor_proto = tf.make_tensor_proto(example_list_with_context_proto, dtype=tf.string, shape=[1])
timeout_in_secs = 3
request = predict_pb2.PredictRequest()
request.inputs['input_ranking_data'].CopyFrom(tensor_proto)
request.model_spec.signature_name = 'predict'
request.model_spec.name = 'tf_ranking_v10'
channel = grpc.insecure_channel("0.0.0.0:8500")
stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)
grpc_response = stub.Predict(request, timeout_in_secs)
unpacked_grpc_response = MessageToDict(grpc_response, preserving_proto_field_name = True)
print(unpacked_grpc_response['outputs']['output']['float_val'])
@azagniotov Thanks for sharing your experience in successfully serving the ranking model with serialized ELWC inputs~ This will be a useful reference for others trying to do the same.
Hi @azagniotov , sorry to re-open this closed issue. I have tried to implement the gRPC response prediction from the gRPC request you detailed above, but I get an error message:
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.INVALID_ARGUMENT details = "Could not parse example input, value: '
Could you please share the exact content of:
""" examples { .... } context { ..... } """
so that I can attempt to reproduce your exact code? Thanks.
@davidmosca sure. give me a day, pls
Nah, did it sooner, @davidmosca :
examples {
features {
feature {
key: "feature_name_a"
value {
float_list {
value: 0.46671828627586365
}
}
}
feature {
key: "feature_name_b"
value {
float_list {
value: 11.34334447979927063
}
}
}
}
}
examples {
features {
feature {
key: "feature_name_a"
value {
float_list {
value: 0.34334447979927063
}
}
}
feature {
key: "feature_name_b"
value {
float_list {
value: 12.032746315002441
}
}
}
}
}
context {
features {
feature {
key: "your_context_feature_name"
value {
float_list {
value: 123.456
}
}
}
}
}
I hope this helps
Thanks for that. Unfortunately I still see the same error, which is strange because I just copied/pasted your code and replaced the variables' values where appropriate (my model too is using ELWC data). Also I know the serving API is up and running because I can make regress cURL requests to it. Do you have any idea in your experience of what else could cause this error?
@davidmosca I am not sure why this is happening, TBH. But, if you could post here your complete code (parsing the ELWC, making the gRPC request, etc), I think it could help.
Here it is:
TF framework module versions
tensorflow-serving-api==2.3.0
tensorflow==2.3.0
tensorflow-ranking==0.3.0
protobuf==3.13.0
grpcio==1.31.0
Some training parameters
_CONTEXT_FEATURES = {'what_tokens'}
_DOCUMENT_FEATURES = {'title_tokens', 'description_tokens', 'relevance'}
_DATA_FORMAT = tfr.data.ELWC
_PADDING_LABEL = -1
gRPC request
import grpc
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc
from tensorflow_serving.apis import input_pb2
from google.protobuf import text_format
from google.protobuf.json_format import MessageToDict
import tensorflow as tf
EXAMPLE_LIST_WITH_CONTEXT_PROTO = text_format.Parse(
"""
context {
features {
feature {
key: "what_tokens"
value {
bytes_list {
value: ["this", "is", "a", "relevant", "question"]
}
}
}
}
}
examples {
features {
feature {
key: "title_tokens"
value {
bytes_list {
value: ["this", "is", "a", "relevant", "answer"]
}
}
}
feature {
key: "description_tokens"
value {
bytes_list {
value: ["with", "a", "relevant", "description"]
}
}
}
feature {
key: "relevance"
value { float_list { value: 5.0 } }
}
}
}
""", input_pb2.ExampleListWithContext())
example_list_with_context_proto = EXAMPLE_LIST_WITH_CONTEXT_PROTO.SerializeToString()
tensor_proto = tf.make_tensor_proto(example_list_with_context_proto, dtype=tf.string, shape=[1])
timeout_in_secs = 5
request = predict_pb2.PredictRequest()
request.inputs['examples'].CopyFrom(tensor_proto)
request.model_spec.signature_name = 'predict'
request.model_spec.name = 'tfr'
channel = grpc.insecure_channel("localhost:8500")
stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)
grpc_response = stub.Predict(request, timeout_in_secs)
unpacked_grpc_response = MessageToDict(grpc_response, preserving_proto_field_name = True)
print(unpacked_grpc_response['outputs']['output']['float_val'])
The request generates this error:
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.INVALID_ARGUMENT details = "Could not parse example input, value: ' 8 description_tokens"
with a relevant description relevance @ 1 title_tokens! this is a relevant answer6 4 2 what_tokens# ! this is a relevant question' [[{{node ParseExample/ParseExampleV2}}]]" debug_error_string = "{"created":"@1599211015.577000000","description":"Error received from peer ipv6:[::1]:8500","file":"src/core/lib/surface/call.cc","file_line":1062,"grpc_message":"Could not parse example input, value: '\n\u0087\u0001\n\u0084\u0001\n8\n\u0012description_tokens\u0012"\n \n\u0004with\n\u0001a\n\brelevant\n\u000bdescription\n\u0015\n\trelevance\u0012\b\u0012\u0006\n\u0004\u0000\u0000\u00a0@\n1\n\ftitle_tokens\u0012!\n\u001f\n\u0004this\n\u0002is\n\u0001a\n\brelevant\n\u0006answer\u00126\n4\n2\n\u000bwhat_tokens\u0012#\n!\n\u0004this\n\u0002is\n\u0001a\n\brelevant\n\bquestion'\n\t [[{{node ParseExample/ParseExampleV2}}]]","grpc_status":3}"
The underlying TF-Ranking model's structure matches that of the EXAMPLE_LIST_WITH_CONTEXT_PROTO above (I have tried with and without relevance
).
TF-Ranking model's predict signature
signature_def['predict']: The given SavedModel SignatureDef contains the following input(s): inputs['examples'] tensor_info: dtype: DT_STRING shape: (-1) name: input_example_tensor:0 The given SavedModel SignatureDef contains the following output(s): outputs['output'] tensor_info: dtype: DT_FLOAT shape: (-1, -1) name: groupwise_dnn_v2/accumulate_scores/div_no_nan:0 Method name is: tensorflow/serving/predict
The Docker dashboard shows that the serving API itself is up and running. Also, a cURL (regress) request to a twin TF Serving API on 8501 (the escaped double quotes are required on Windows):
curl -d "{\"context\": {\"what_tokens\": [\"a\", \"question\"]}, \"examples\": [{\"title_tokens\": \"answer\", \"description_tokens\": \"description\"}]}" -X POST http://localhost:8501/v1/models/tfr:regress
produces this response:
{
"results": [-10.4137278]
}
@davidmosca
I took your code as-is and ran it. I was able successfully (i.e.: I did not observe your aforementioned error and got back a list of predictions) to make a request to the Docker container over gRPC using TF/TFR v2.1.0
and also using v2.3.0
as another attempt, which is the versions you are on.
Are you pinning down the docker container version when pulling it down or are you getting the latest? i.e.: docker pull tensorflow/serving:2.3.0
or docker pull tensorflow/serving
? The latter will pull down the latest
tag, which may or may not be what you want. I remember some time ago when I was not pinning down the container version and one day things stopped working for me because I probably pulled down new functionality or changes from the upstream. So, I had to fix the Docker version.
In my Docker start command, I also explicitly start the version that I am on, e.g.:
docker run -t --rm -p 8500:8500 -p 8501:8501 \
-v "/home/azagniotov/tensorflow_ranking_model:/models/tensorflow_ranking_model" \
-e MODEL_NAME="tensorflow_ranking_model" tensorflow/serving:2.1.0 &
tensorflow-serving-api==2.1.0 (or 2.3.0)
tensorflow==2.1.0 (or 2.3.0)
tensorflow-ranking==0.3.0
docker pull tensorflow/serving:2.1.0 (or 2.3.0)
In the following code I am using to generate ranking_serving_input_receiver_fn
when saving the model. Not sure if this helps, but I thought to show how I do it:
# Dicts, e.g.:
# spec['feature_name_a'] = tf.feature_column.numeric_column('feature_name_a', shape=[1], default_value=-1, dtype=tf.float32)
context_feature_spec = tf.feature_column.make_parse_example_spec(context_feature_columns().values())
example_feature_spec = tf.feature_column.make_parse_example_spec(example_feature_columns().values())
# I am using `input_ranking_data` name as the `receiver_name`
serving_input_receiver_fn = tfr.data.build_ranking_serving_input_receiver_fn(
data_format=tfr.data.ELWC,
list_size=_LIST_SIZE,
default_batch_size=None,
size_feature_name='example_list_size',
receiver_name='input_ranking_data',
context_feature_spec=context_feature_spec,
example_feature_spec=example_feature_spec)
# ranker == tf.estimator.Estimator
ranker.export_saved_model(export_dir_base='tensorflow_ranking_model',
serving_input_receiver_fn=serving_input_receiver_fn,
assets_extra={'tf_serving_warmup_requests': 'tf_serving_warmup_requests'},
as_text=False,
checkpoint_path=None,
experimental_mode=tf.estimator.ModeKeys.PREDICT)
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['input_ranking_data'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_ranking_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['output'] tensor_info:
dtype: DT_FLOAT
shape: (-1, -1)
name: groupwise_dnn_v2/accumulate_scores/div_no_nan:0
Method name is: tensorflow/serving/predict
Nah, did it sooner, @davidmosca :
examples { features { feature { key: "feature_name_a" value { float_list { value: 0.46671828627586365 } } } feature { key: "feature_name_b" value { float_list { value: 11.34334447979927063 } } } } } examples { features { feature { key: "feature_name_a" value { float_list { value: 0.34334447979927063 } } } feature { key: "feature_name_b" value { float_list { value: 12.032746315002441 } } } } } context { features { feature { key: "your_context_feature_name" value { float_list { value: 123.456 } } } } }
I hope this helps
Hello, @azagniotov, could you please clarify, how to parse such ELWC with tfr.data.build_ranking_dataset, is it ok to use such function?
def create_feature_columns():
sparse_column = tf.feature_column.categorical_column_with_hash_bucket(
key="your_context_feature_name", dtype= tf.string, hash_bucket_size=1)
query_embedding_column = tf.feature_column.embedding_column(
sparse_column, dimension=1)
context_feature_columns = {"your_context_feature_name": query_embedding_column}
example_feature_columns = {}
for fe in ['feature_name_a', 'feature_name_a']:
example_feature_columns[str(fe)] = tf.feature_column.numeric_column( str(fe), dtype=tf.float32)
return context_feature_columns, example_feature_columns
def make_dataset(file_pattern,
batch_size,
randomize_input=False,
num_epochs=None):
context_feature_columns, example_feature_columns = create_feature_columns()
context_feature_spec = tf.feature_column.make_parse_example_spec(
context_feature_columns.values())
label_column = tf.feature_column.numeric_column(
'target_name', dtype=tf.float32, default_value= -1)
example_feature_spec = tf.feature_column.make_parse_example_spec(
list(example_feature_columns.values()) + [label_column])
dataset = tfr.data.build_ranking_dataset(
file_pattern=file_pattern,
data_format=tfr.data.ELWC,
batch_size=batch_size,
context_feature_spec=context_feature_spec,
example_feature_spec=example_feature_spec,
reader=tf.data.TFRecordDataset,
shuffle=randomize_input,
num_epochs=num_epochs,
size_feature_name=None)
Thank you in advance, your comments really helpful.
OK I found what the problem was. When you define the example feature_columns in the TF-Ranking model, you always need to assign a dummy default value, for instance:
title_column = tf.feature_column.categorical_column_with_vocabulary_file(key="title_tokens", vocabulary_file=FLAGS.vocab_path, default_value=0)
This is because the lists of examples are of varying length, and not setting a default value creates an issue which is not identified as such in the error message. Neither the Colab example nor the TFRecord example contains such default value (or at least the requirement for such value) so that's not helpful. The lack of TF-Ranking documentation is not helpful either.
Interestingly, adding this default value also automatically changes the input tensor alias from examples
to input_ranking_data
.
@azagniotov, thanks for sharing the gRPC request, that was very helpful in setting up TensorFlow Serving for online inference.
@davidmosca No worries. I am glad I could help. I edited my reply in the last few minutes and added some information about pinning down the Docker container version, I am not sure if you saw it.
I saw it just after publishing mine :) Thanks again.
Hi @utopn , I do not think I understand your question:
Hello, @azagniotov, could you please clarify, how to parse such ELWC with tfr.data.build_ranking_dataset, is it ok to use such function?
In order to build a ranking dataset, you need to provide to the tfr.data.build_ranking_dataset
a path (i.e.: file_pattern
arg) in the local filesystem where your TF record(s) that contains your ELWCs are located. So, if you have not done so, first you need to generated TF record(s) with ELWCs
I am closing this issue, @utopn as your question is out of scope here. Feel free to raise a new issue to address your question specifically
Hello Team,
I trained a TF ranking model (basing my training on the following example: https://github.com/tensorflow/ranking/blob/master/tensorflow_ranking/examples/tf_ranking_tfrecord.py) and saved it using
estimator.export_saved_model('my_model', serving_input_receiver_fn)
, the model was trained successfully & saved without any warnings/errors.I deployed the model to a local TensorFlow ModelServer and made an call to it over HTTP using cURL as described on https://www.tensorflow.org/tfx/serving/api_rest#request_format. Unfortunately I see the following error after making the request:
I understand that this is a problem that may be related to serialization where my input was not properly serialized, but saving the model by generating
serving_input_receiver_fn
& using it produced no errors/warnings, so I am not sure where to start looking to resolve this.I am providing some details below, please let me know if you need more information.
Details
TF framework module versions
tensorflow-serving-api
==2.0.0tensorflow
==2.0.0tensorflow-ranking
==0.2.0Some training parameters and functions
_CONTEXT_FEATURES
={'ctx_f0'}
_DOCUMENT_FEATURES
={'f0', 'f1', 'f2'}
_DATA_FORMAT
=tfr.data.ELWC
_PADDING_LABEL
=-1
Creating the serving_input_receiver_fn
When making a REST API to a local TensorFlow ModelServer using the following cURL request
The error is as follows: