Why can't the frame rate reach 77fps in local computer test

[zhishanz]

Why can't the frame rate reach 77fps in local computer test, using 3090gpu, reasoning speed is 0.3-0.5s per graph（288*288）

[DonaldRR]

There are a few things you need to make sure during inference:

1. torch.no_grad anywhere
2. clip generator's code
3. counting time except the first inference (cuda memory allocation for the first inference takes time)
4. counting time with cuda synchronization (though it does add time)

Infering with TensorRT could be much faster.

(Composition of infering time: backbone infer time + classifier infer time + post-process time. The backbone inference takes the majority of inference time, and it is ~10ms)

[code10086web]

It is unfair to compare the inference time of TensorRT with PatchCore, as TensorRT is not utilized in the latter.