depth encoding time benchmark

[bmegli]

Benchmark time needed for encoding to HEVC Main10 including upload to hardware encoding, download from hardware.

Related to hardware-video-streaming#4

[bmegli]

Overview

The current program structure is simple sequential (grab data, upload to hardware, hardware encode, download from hardware, send). Some of those operations could be made to run concurrently if needed.

Methodology

Benchmark 56bb454 with NHVE@432cfcb:

• simple measurement of every upload, encode, download
• followed with printf

Just to get idea of order of execution time

[bmegli]

Setting

• 848x480 depth data
• i7-7820HK Laptop (KabyLake)

 ./realsense-nhve-hevc 192.168.0.100 9768 depth 848 480 30 500 /dev/dri/renderD128 80000000 0.000025

Results

• upload time seems to be negligible (microseconds)
  • I would have to double check if the upload time was correctly measured
• first frame encoding takes around 15 ms
• following frames take 7 to 9 ms

[bmegli]

Setting

• 848x480 depth + infrared data
• i7-7820HK Laptop (KabyLake)

 ./realsense-nhve-hevc 192.168.0.100 9768 depth+ir 848 480 30 500 /dev/dri/renderD128 80000000 0.000025

Results

No noticeable difference between depth and depth+ir in hardware encoding time (as above).

[bmegli]

Setting

• 640x360 color data
• i7-7820HK Laptop (KabyLake)

 ./realsense-nhve-hevc 192.168.0.100 9768 color 640 360 30 500 /dev/dri/renderD128 80000000

Results

• first frame encoding takes around 10-14 ms
• following frames take 5 to 6 ms

[bmegli]

Setting

• 1920x1080 color data
• i7-7820HK Laptop (KabyLake)

 ./realsense-nhve-hevc 192.168.0.100 9768 color 1920x1080 30 500 /dev/dri/renderD128 80000000

Results

• first frame encoding takes around 20-35 ms
• following frames take 15 to 17 ms

[bmegli]

Setting

• 848x480 infrared data
• i7-7820HK Laptop (KabyLake)

 ./realsense-nhve-hevc 127.0.0.1 9768 infrared 848 480 30 500 /dev/dri/renderD128 80000000

Results

• first frame encoding takes up to 15 ms
• following frames take 5 to 8 ms

[bmegli]

As above but with 1 Mb bitrate no noticable differences

[bmegli]

Now H.264

[bmegli]

Setting

• 640x360 color data
• i7-7820HK Laptop (KabyLake)

./realsense-nhve-h264 127.0.0.1 9766 color 640 360 30 5 /dev/dri/renderD128 2000000

Results

• first frame encoding takes 6-7 ms
• following frames take 3 to 4 ms

[bmegli]

Setting

• 1920x1080 color data
• i7-7820HK Laptop (KabyLake)

./realsense-nhve-h264 127.0.0.1 9766 color 1920 1080 30 5 /dev/dri/renderD128 2000000

Results

• first frame encoding takes up to 25 ms
• following frames take 9 to 10 ms

[bmegli]

Setting

• 848x480 infrared data
• i7-7820HK Laptop (KabyLake)

./realsense-nhve-h264 127.0.0.1 9766 infrared 848 480 30 50 /dev/dri/renderD128 1000000

Results

• first frame encoding takes up to 8 ms
• following frames take 3 to 5 ms

[bmegli]

Setting

• 640x360 infrared data
• i7-7820HK Laptop (KabyLake)

./realsense-nhve-h264 127.0.0.1 9766 infrared 640 360 30 50 /dev/dri/renderD128 1000000

Results

• first frame encoding takes up to 7 ms
• following frames take 3 to 4 ms

[bmegli]

Now LattePanda Alpha m3-7y30

[bmegli]

Setting

• 848x480 depth data
• LattePanda Alpha m3-7y30 (KabyLake)

./realsense-nhve-hevc 127.0.0.1 9768 depth 848 480 30 500 /dev/dri/renderD128 8000000 0.0000125

Results

• first frame encoding takes up to 18 ms
• following frames take 8 to 10 ms

[bmegli]

Setting

• 640x360 depth data
• LattePanda Alpha m3-7y30 (KabyLake)

./realsense-nhve-hevc 127.0.0.1 9768 depth 640 360 30 500 /dev/dri/renderD128 8000000 0.0000125

Results

• first frame encoding takes up to 15 ms
• following frames take 6 to 7 ms

[bmegli]

Depth+ir no noticeable difference from depth only

[bmegli]

Setting

• 848x480 infrared data
• LattePanda Alpha m3-7y30 (KabyLake)

./realsense-nhve-hevc 127.0.0.1 9766 infrared 848 480 30 50 /dev/dri/renderD128 1000000

Results

• first frame encoding takes up to 17 ms
• following frames take 6 to 8 ms

[bmegli]

Setting

• 640x360 infrared data
• LattePanda Alpha m3-7y30 (KabyLake)

./realsense-nhve-hevc 127.0.0.1 9766 infrared 640 360 30 50 /dev/dri/renderD128 1000000

Results

• first frame encoding takes up to 15 ms
• following frames take 4 to 7 ms

[bmegli]

Now H264

[bmegli]

Setting

• 848x480 infrared data
• LattePanda Alpha m3-7y30 (KabyLake)

/realsense-nhve-h264 127.0.0.1 9766 infrared 848 480 30 5 /dev/dri/renderD128 1000000

Results

• first frame encoding takes up to 9 ms
• following frames take 3 to 5 ms

[bmegli]

Setting

• 640x360 infrared data
• LattePanda Alpha m3-7y30 (KabyLake)

/realsense-nhve-h264 127.0.0.1 9766 infrared 640 360 30 5 /dev/dri/renderD128 1000000

Results

• first frame encoding takes up to 7 ms
• following frames take 3 to 4 ms

[bmegli]

Summary

scenario	i7-7820hk	LPA m3-7y30
848x480 depth HEVC Main10	7-9 ms	8-10 ms
640x360 depth HEVC Main10	5-6 ms	6-7 ms
848x480 ir HEVC Main	5-8 ms	6-8 ms
848x480 ir H264	3-5 ms	3-5 ms
640x360 ir H264	3-4 ms	3-4 ms

• first frame may take up to 2.5 times longer
• time increases with resolution

[bmegli]

Discussion

Encoding at resolution up to 848x480 takes less than 10 ms in all tested scenarios.

This means that:

• around 30 fps the current simple sequential scenario should be ok
• around 90 fps optimization would be mandatory
• around 60 fps (~ 16.66 ms) probably ok but depends how much time other operations take

Side note - Full HD HEVC would need optimizations for 60 fps (15-17 ms encoding time).

[bmegli]

The above benchmark was with default ffmpeg/VAAPI compression level.

It is possible to specify quality/time trade-off with compression_level.

Notes:

• the default value when unset is -1.
• by setting it to 0 no difference in intel_gpu_top and timings (0 is probably driver default)
• by setting it to 1 I observe:
  • rise of render busy from 15% to 25%
  • rise of encoding time from 7-9 ms to 10-12 ms

With i7-7820HK Laptop (KabyLake) and:

./realsense-nhve-hevc 192.168.0.100 9768 depth 848 480 30 500 /dev/dri/renderD128 80000000 0.000025

This suggests that compression_level should be exposed as option, see HVE#6