search

emacski / tensorflow-serving-arm

TensorFlow Serving ARM - A project for cross-compiling TensorFlow Serving targeting popular ARM cores
Apache License 2.0
100 stars 16 forks source link

No Prometheus metrics in serving? #15

Open pherna06 opened 2 years ago

pherna06 commented 2 years ago

Hello, @emacski .

I am using your repository to implement a TFServing in a RaspberryPi, but apart from being able to make inferences over the models in the serving, I need to obtain the serving metrics too. In the original tensorflow/serving repository, this metrics are exposed as Prometheus metrics (with metrics path in '/monitoring/prometheus/metrics'), but they don't seem to be available when using the ARM serving of this repository.

I've been diving in your repository for a while, but I don't really understand how you make it work: i would suppose you are implementing the same 'tensorflow_model_server' that is used as entrypoint script in the original tensorflow/serving Dockerfiles, but then why do the metrics seem to be unavailable?

Thank you in advance.

pherna06 commented 2 years ago

I already figured out how to build the ARM images from the devel image (sorry that I could not completely understand the guide when I wrote the last comment). So, it seems that Prometheus metrics were not implemented in TF and TFServing version 2.6.0 yet. So I tried to rewrite the Bazel files to use the most recent version 2.8.0 (mainly the WORKSPACE file), but there seems changes in 'patches' for Tensorflow between 2.6.0 and 2.8.0, which I don't really know how to assess.

I would really appreciate if anyone could give me some help with it.