ffmpeg performance on graviton3 is not as good as x86

[xshen053]

Hi, I used phoronix-test-suite to benchmark performance on x86 and graviton3 instance. This test suite uses vbench to benchmark performance of ffmpeg. I used c6a.8xlarge and c7g.8xlarge instances. However the result is not expected. Maybe I did something wrong?

Scenario

Encoder: libx264 - Scenario: Video on demand

c6a.8xlarge

FPS: 44.6 seconds: 170

c7g.8xlarge

FPS: 30.64 seconds: 247

From this blog, seems graviton has some optimization

https://aws.amazon.com/blogs/opensource/optimized-video-encoding-with-ffmpeg-on-aws-graviton-processors/

[snadampal]

Hi @xshen053 , please share the below details to better understand the scenario

1. ffmpeg version you are using
2. whether you are building them from sources or using the release binaries
3. which OS distribution
4. ffmpeg commandline being used in the benchmarking or exact commands you used for running the vbench.

[xshen053]

ffmpeg

http://ffmpeg.org/releases/ffmpeg-6.0.tar.xz

x264

http://www.phoronix-test-suite.com/benchmark-files/x264-20221005.tar.xz

OS distribution

Ubuntu 20.04.6 LTS

exact commands I used for running the vbench

install phoronix-test-suite

git clone https://github.com/phoronix-test-suite/phoronix-test-suite.git sudo ./install-sh might need install other dependency like

sudo apt install php7.4-cli
sudo apt-get install php-xml

execute benchmark

phoronix-test-suite benchmark ffmpeg choose 1

choose 4

then it will automatically execute vbench benchmark and give me results after finishing.

[xshen053]

hey, can you run the test I did, do you need any other informations?

[snadampal]

Hi @xshen053 , we are trying to reproduce your scenario. Can you also please let's know the CPU utilization you observed during the runs? is it possible to use your real application for benchmarking the performance instead ?

[xshen053]

• thanks, can you tell me what kind of CPU utilization do you want me to record, is htop enough?
• I actually don't have a real application using ffmpeg, do you have any recommendation, I don't mind to benchmark again, just this test suite provides benchmark for ffmpeg, so that's why I use it, for a real application, if it doesn't have benchmark, I am not sure if the way I benchmark the performance is correct, is it possible if you can recomemend some real application repo, I can benchmark them

[AWSjswinney]

I tried to run the test but I ran into a problem with the phoronix code so I wasn't able to run it without investing some time to debug. By guess would be that the test is single threaded (or at least leaving many cores idle) which would explain the performance discrepancy. Graviton CPUs in general are optimized to sustain large workloads over many (or all) cores without reductions in performance. Other CPUs that have SMT can encounter resource constraints when fully loading across the whole instance. There is some data about that here: https://github.com/aws/aws-graviton-getting-started/blob/main/perfrunbook/system-load-and-compute-headroom.md

As mentioned in the blog post that you linked to, most video workloads utilize entire instances in order to transcode many video streams or files in parallel. This leads to lowest cost per unit time of video. When I ran the benchmarks for that post, I designed my benchmarks to fully load the instances. In that scenario, Graviton3 powered c7g instances achieved the lowest cost to encode of the instances I tested, which were C6i, C6a, and C7g.

I suspect that there is some step that you ran that is missing which is preventing me from running the phoronix test. (That would ultimately mean there's a bug in the test.) Perhaps libx264-devel was already installed from the Ubuntu package manager that led to ffmpeg building with an older (and less optimized) version? (Just a guess...)

[AWSjswinney]

A rough approximation of the CPU usage with htop would be fine. If you want to explore a more rigorous method, use sysstat. Just make sure you get an idea for how the usage changes during the test. E.g. does it start out using one core and then use all in the middle? Is it steady state or periodic?

[xshen053]

Thanks for the reply!

• Btw, I think your guess is correct, I checked the cpu using htop, and when that test is running, it is single threaded! So can I get the conclusion that graviton doesn't perform as good as x86 if an application uses only one thread? (rare in real production application though)
• I will double check and update the steps and I hope you can rerun it again!!

[AWSjswinney]

Everything depends on the workload and you would need to benchmark for what you are interested in, but it can be the case that a single thread will run faster on an M7i, M6i, or C6a than C7g, as you have seen with this ffmpeg benchmark.

[geoffreyblake]

Closing this issue as the question appears to have been answered.