serizba / cppflow

Run TensorFlow models in C++ without installation and without Bazel
https://serizba.github.io/cppflow/
MIT License
779 stars 177 forks source link

Support for signature names instead of tensor names #203

Open CarlPoirier opened 2 years ago

CarlPoirier commented 2 years ago

Hi folks,

So I wanted to use cppflow for models with multiple outputs that I can differentiate only by their names instead of the tensor names which are all "StatefulPartitionedCall:index". The way I proceed is quite ugly, but that's what I was able to come up with in a short amount of time. I'm opening a pull request because I'd like to gather comments.

For example, this is the output of saved_model_cli.exe : image You would call model::get_tensor_mapping("detection_scores") and it would return "StatefulPartitionedCall:4". Then, you could use this as usual to get the detection scores.

The mapping is present in the metagraph which is extracted with the tensorflow C API, but not parsed. AFAIK, the latter just does not have the functionality to do so, so adding it in cppflow is somewhat sloppy. I'm searching for the raw bytes to extract it.

serizba commented 1 year ago

Hi @CarlPoirier

This is definitely interesting, since the beginning of cppflow I've been bothered with the StatefulParitionedCall names.

So, if I understood correctly, you are parsing the protobuf MetaGraphDef to obtaining the correspondences between names and tensor names. As from the code is a bit difficult to tell how you are doing the parsing, could you elaborate a bit more on what are you trying to find?

I think directly parsing the MetaGraphDef as a string is not optimal, and may be a problem if the protocols change. Do you know any alternative to this? I've read the C API but I did not found any function providing information about MetaGraphDef. Maybe opening an issue on the TensorFlow repo we may obtain some info on how to proceed with this.

Thanks for your work!

CarlPoirier commented 1 year ago

I opened an issue on the TF repo. We'll see!

serizba commented 1 year ago

Thanks, let's see what they say