The KServe spec for gRPC uses a field called raw_*_content to manage data that can't be easily represented by native types in gRPC (e.g. FP16). For such fields, the inference server currently converts the type to/from the closest valid type when data is converted from/to gRPC. Using the raw content field can allow native processing of these data types but it needs special handling logic on the server since the data is always assumed to be part of the InputTensor rather than in the request itself.
The KServe spec for gRPC uses a field called
raw_*_contentto manage data that can't be easily represented by native types in gRPC (e.g. FP16). For such fields, the inference server currently converts the type to/from the closest valid type when data is converted from/to gRPC. Using the raw content field can allow native processing of these data types but it needs special handling logic on the server since the data is always assumed to be part of the InputTensor rather than in the request itself.