search

triton-inference-server / dali_backend

The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.
https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html
MIT License
121 stars 28 forks source link

Prepare automation to benchmark DALI vs Python #119

Closed szalpal closed 2 years ago

szalpal commented 2 years ago

This is first part of benchmarking pipelines: DALI vs Python-backend-based. This commit prepares outline of the setup, not automating anything for now. The benchmark is conducted via perf_analyzer. I'm assuming, that the inputs to the both pipelines under test are the same.

The final solution would consist of two general functionalities:

  1. Benchmarking the pipelines,
  2. Verifying output consistency.

Due to nature of the processing and differences in the implementation, we can't assume, that the outputs will be bit-exact. For now I'm focusing on simplifying the manual verification. In the RN50 case, the images are saved to the file, so they can by easily viewed from numpy.

As I stated above, this is the first PR out of a couple, so there are still hard-coded parts, which I will be changing at the later stage.

Signed-off-by: szalpal mszolucha@nvidia.com

dali-automaton commented 2 years ago

CI MESSAGE: [4151689]: BUILD STARTED

dali-automaton commented 2 years ago

CI MESSAGE: [4151689]: BUILD PASSED