Open vvolhejn opened 2 years ago
Hi vvolhejn, sorry for the late reply, did you figure out this issue yet?
For your 2nd question, some observations:
_r=1
. fake_batch = tf.random.normal((8, 32))
line underneath the tf.profiler.experimental.Trace scopeFor your 1st question, not sure why your Type and Op are the same. noticed that you seem to use the Context manager API
to collect the profile data, be sure to use it only in local mode (having the API called in the same machine where your model is running, not remotely like a cloud machine). If it's still not working for you, maybe try switching to the tf.profiler.experimental start/stop API
instead and try again?
Hi, I'm trying to profile the inference performance of my model on CPU to figure out where the bottlenecks are. However, when I run the profiler and open tensorflow_stats, it groups all the operations by type (MatMul, BiasAdd...) and doesn't tell me which layer is taking how much time.
Here is a minimal example:
The console output doesn't report anything weird:
When I view the results in Tensorboard,, I get:
Here I would like the two layers to be shown as separate elements in the charts, rather than there being a single "MatMul" element for both matrix multiplications together. This is also what the guide shows in one of the screenshots. How can I achieve this?
Furthermore, when I run this code, I can only see tensorflow_stats and trace_viewer; some of the others say "No step marker observed and hence the step time is unknown". Am I using the profiler wrong? My model uses a custom training loop so I am using context managers rather than a callback, as per the guide.
Versions: tensorflow 2.8.0 tensorboard 2.8.0 tensorboard_plugin_profile 2.5.0 macOS Catalina 10.15.7