Profiling with Keras 3

[tgpsantos]

Hi there!

I'd like to understand what's the recommended approach to Profile a model. For instance, I'd like to understand which ops take more time when running (even for a Forward Pass)

This is what I've tried

1. Backend Tensorflow (TF Profiler & Tensorboard)

Created a Keras model w/ Tensorflow backend and then used tf.profiler.

Problem here is that tensorboard does not show relevant info, just shows No step marker observed and hence the step time is unknown. This may happen if (1) training steps are not instrumented (e.g., if you are not using Keras) or (2) the profiling duration is shorter than the step time. For (1), you need to add step instrumentation; for (2), you may try to profile longer.

Here's a working Notebook https://colab.research.google.com/drive/1dcKJfyFWQSWsB4QzTH4YNo5TDCxHOWjG?usp=sharing

2. Backend Pytorch (Torch Profiler)

I've done similar, but with the nuance of changing the pytorch backend and using import torch.autograd.profiler as profiler and doing the forward passes inside of profiler.profile

Even though I get a clearer table with results, CPU times, etc, there's no way to distinguish the ops by the names. For instance, all the Conv2D show up as the same row, no matter their position in the model.

Here's the working Notebook: https://colab.research.google.com/drive/1uiH5oaRfCtxQG6Zd440PMW63kqpToKPZ?usp=sharing

---

What's the recommended Keras way to do a profiling of a model or training? (without using model.fit)

Did I miss something on the previous shared notebooks?

[innat]

Check this discussion. https://github.com/keras-team/keras/issues/19023

[ghsanti]

this may not turn out being useful but you can checkout this example https://github.com/keras-team/keras/issues/19023#issuecomment-2264486340