This issue serves as a shared note to track the status of SavedModel C API in TF 2.0, and also for discussions on possible implementations in cppflow2.
The original RFC proposal is here. The loading functions are defined here
However, as of this writing, the new SavedModel C API in TF 2.0 is still work-in-progress. It is not enabled in bazel. To enable the API, the following patch is needed. In addition, headers need to be copied manually. Although this patch enables the experimental APIs in the libtensorflow, calling these APIs will throw unimplemented errors. (at least in v2.3) I saw commits were submitted to support the new SavedModel C API in v2.4, so some of them may work now.
There seems to be two ways to call the model, ConcreteFunction (TF_GetSavedModelConcreteFunction) and SignatureDefFunction (TF_GetSavedModelSignatureDefFunction) defined in this header, but both of them return TFE_Op* (TF_SignatureDefFunctionMakeCallOp and TF_ConcreteFunctionMakeCallOp). They should be compatible with eager execution in cppflow2.
Additional notes
The ConcreteFunction signature may involve high-level python class, e.g. RaggedTensorSpec for RaggedTensor. They may require significant work to be implemented in C++. The corresponding SignatureDefFunction is rather low-level, where all the Tensors are flattened, e.g. the prevous RaggedTensor is represented by two flat Tensors. From this perspective, starting with SignatureDefFunction seems to be a good idea.
Edit:
ConcreteFunction (TF_GetSavedModelConcreteFunction) implementation seems to be available on the master branch.
SignatureDefFunction (TF_GetSavedModelSignatureDefFunction) is not implemented yet.
Thanks @ljn917 @serizba
I stumbled into this thread, after failing alot to use c/cc tensorflow 2.x APIs.
Your work here gave me lots of good insights.
Please keep it up
This issue serves as a shared note to track the status of SavedModel C API in TF 2.0, and also for discussions on possible implementations in cppflow2.
The original RFC proposal is here. The loading functions are defined here
However, as of this writing, the new SavedModel C API in TF 2.0 is still work-in-progress. It is not enabled in bazel. To enable the API, the following patch is needed. In addition, headers need to be copied manually. Although this patch enables the experimental APIs in the libtensorflow, calling these APIs will throw unimplemented errors. (at least in v2.3) I saw commits were submitted to support the new SavedModel C API in v2.4, so some of them may work now.
There seems to be two ways to call the model,
ConcreteFunction
(TF_GetSavedModelConcreteFunction
) andSignatureDefFunction
(TF_GetSavedModelSignatureDefFunction
) defined in this header, but both of them returnTFE_Op*
(TF_SignatureDefFunctionMakeCallOp and TF_ConcreteFunctionMakeCallOp). They should be compatible with eager execution in cppflow2.Additional notes
ConcreteFunction
signature may involve high-level python class, e.g.RaggedTensorSpec
forRaggedTensor
. They may require significant work to be implemented in C++. The correspondingSignatureDefFunction
is rather low-level, where all theTensor
s are flattened, e.g. the prevousRaggedTensor
is represented by two flatTensor
s. From this perspective, starting withSignatureDefFunction
seems to be a good idea.Edit:
ConcreteFunction
(TF_GetSavedModelConcreteFunction
) implementation seems to be available on the master branch.SignatureDefFunction
(TF_GetSavedModelSignatureDefFunction
) is not implemented yet.Edit2: A WIP for the TF v2 SavedModel API in cppflow: https://github.com/ljn917/cppflow/tree/cppflow2-savedmodel
Known limitations:
tf.function
.