arm64 failure to run any benchmark

[osingla]

Can these benchmarks run on arm64 system? Using fedora35 on arm64 5.16.12-200.fc35.aarch64 I always get this kind of error:

==> Running benchmark for go_grpc_bench... Waiting for server to come up... ready. Warming up the service for 5s... WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested standard_init_linux.go:228: exec user process caused: exec format error done. Benchmarking now... WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested standard_init_linux.go:228: exec user process caused: exec format error done. Results:

---

Benchmark finished. Detailed results are located in: results/221103T105314 /analyze/results_analyze.rb:18:in block in <main>': undefined method[]' for nil:NilClass (NoMethodError) from /analyze/results_analyze.rb:15:in each' from /analyze/results_analyze.rb:15:in

' Failed miserably analysing go_grpc. Analysis fiascoed. total 12K drwxr-xr-x 2 root root 4.0K Mar 11 10:53 . drwxr-xr-x 8 root root 4.0K Mar 11 10:53 .. -rw-r--r-- 1 root root 0 Mar 11 10:53 go_grpc_bench.report -rw-r--r-- 1 root root 56 Mar 11 10:53 go_grpc_bench.stats

results/221103T105314/go_grpc_bench.report

results/221103T105314/go_grpc_bench.stats 0.00% 2.758MiB / 512MiB 0.00% 2.758MiB / 512MiB

[LesnyRumcajs]

Hello @osingla . In general probably they are not processor architecture agnostic, I assume it depends on a particular implementation (e.g. PHP benchmark shamelessly downloads binaries for amd64). Others may be more restrictive.

That being said, the ghz (tool we use to generate traffic) images aren't available for ARM, nor the images generated from the original repository.

It's a very good point though, it would be great to see what are the differences across CPU architectures.

[odidev]

@LesnyRumcajs, I was trying to run the benchmarks for arm64 architecture but I am getting the following error:

Benchmarking now...
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
exec ./entrypoint.sh: exec format error
                done.
                Results:

-----
Benchmark finished. Detailed results are located in: results/221410T083531
/analyze/results_analyze.rb:18:in `block in <main>': undefined method `[]' for nil:NilClass (NoMethodError)
        from /analyze/results_analyze.rb:15:in `each'
        from /analyze/results_analyze.rb:15:in `<main>'
Failed miserably analysing cpp_asio_grpc.
Analysis fiascoed.
total 12K
drwxr-xr-x  2 root   root   4.0K Oct 14 08:35 .
drwxrwxr-x 18 ubuntu ubuntu 4.0K Oct 14 08:35 ..
-rw-r--r--  1 root   root      0 Oct 14 08:35 cpp_asio_grpc_bench.report
-rw-r--r--  1 root   root     54 Oct 14 08:35 cpp_asio_grpc_bench.stats

results/221410T083531/cpp_asio_grpc_bench.report

results/221410T083531/cpp_asio_grpc_bench.stats
0.01%     1.77MiB / 512MiB
0.01%     1.77MiB / 512MiB

In the script bench.sh looks line the error is thrown from the following code snippet:

# Start the gRPC Server container
        docker run \
                --name "${NAME}" \
                --rm \
                --cpus "${GRPC_SERVER_CPUS}" \
                --memory "${GRPC_SERVER_RAM}" \
                -e GRPC_SERVER_CPUS \
                -e GRPC_SERVER_RAM \
                -p 50051:50051 \
                --detach \
                --tty \
                "$GRPC_IMAGE_NAME:${NAME}-$GRPC_REQUEST_SCENARIO" >/dev/null

As I can see the images which are getting used are not present for arm64. Could you please share your pointers on how we can build these iamges for arm64. I would be really happy to raise the PR for the same.

[fenollp]

To build images for other platforms than the local one (thus using QEMU) see: https://github.com/moby/buildkit/blob/master/docs/multi-platform.md Then the docker image manifest should contain the multiple architectures.

[LesnyRumcajs]

This looks interesting, thanks @fenollp. It'd be interesting to see the performance hit when using QEMU (compared to native).

[fenollp]

Ah BuildKit uses QEMU only when building the image. Running containers is still outside the emulator so no performance diff there.

[LesnyRumcajs]

@fenollp That's neat, thanks. BuildKit folks don't encourage building binaries through emulation https://github.com/moby/buildkit/blob/master/docs/multi-platform.md#builds-are-very-slow-through-emulation. Fortunately, our simple gRPC servers are probably simple enough that this wouldn't matter much.

[odidev]

@LesnyRumcajs, I was trying to run the docker file for both architectures (arm64, x86) but I was getting the below error:

ubuntu@master-node:~grpc_bench/java_openj9_grpc_gencon_bench$ docker build .  
Sending build context to Docker daemon  95.23kB   
Step 1/11 : FROM ibm-semeru-runtimes:open-18.0.2_9-jdk  
---> 882b3abde6f4  
Step 2/11 : WORKDIR /app  
---> Using cache  
---> ed6f12ac3782  
Step 3/11 : COPY java_hotspot_grpc_sgc_bench/ /app  
---> Using cache  
---> ad5205492288  
Step 4/11 : COPY populate_scc.sh /app  
---> Using cache  
---> 3c0646980c0d  
Step 5/11 : COPY proto/helloworld/helloworld.proto /app/src/main/proto/helloworld.proto  
COPY failed: file not found in build context or excluded by .dockerignore: stat proto/helloworld/helloworld.proto: file does not exist 

I have created this proto/helloworld/helloworld.proto file by using the mkdir command. After creating the proto directory, I was getting below error:

Starting a Gradle Daemon (subsequent builds will be faster) 

> Task :extractIncludeProto 

> Task :extractProto 

> Task :generateProto 

> Task :compileJava FAILED 

/app/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java:109: error: package GreeterGrpc does not exist 

  static class GreeterImpl extends GreeterGrpc.GreeterImplBase { 

                                              ^ 

/app/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java:112: error: cannot find symbol 

    public void sayHello(HelloRequest req, StreamObserver<HelloReply> responseObserver) { 

                         ^ 

  symbol:   class HelloRequest 

  location: class GreeterImpl 

/app/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java:112: error: cannot find symbol 

    public void sayHello(HelloRequest req, StreamObserver<HelloReply> responseObserver) { 

                                                          ^ 

  symbol:   class HelloReply 

  location: class GreeterImpl 

/app/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java:40: error: no suitable method found for addService(GreeterImpl) 

        .addService(new GreeterImpl()) 

        ^ 

    method ServerBuilder.addService(ServerServiceDefinition) is not applicable 

      (argument mismatch; GreeterImpl cannot be converted to ServerServiceDefinition) 

    method ServerBuilder.addService(BindableService) is not applicable 

      (argument mismatch; GreeterImpl cannot be converted to BindableService) 

  where CAP#1 is a fresh type-variable: 

    CAP#1 extends ServerBuilder<CAP#1> from capture of ? 

/app/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java:111: error: method does not override or implement a method from a supertype 

    @Override 

    ^ 

/app/src/main/java/io/grpc/examples/helloworld/HelloWorldServer.java:113: error: cannot find symbol 

      final var reply = HelloReply.newBuilder().setResponse(req.getRequest()).build(); 

                        ^ 

  symbol:   variable HelloReply 

  location: class GreeterImpl 

6 errors 
FAILURE: Build failed with an exception.   
* What went wrong: 
Execution failed for task ':compileJava'. 
> Compilation failed; see the compiler error output for details. 
* Try: 

> Run with --stacktrace option to get the stack trace. 

> Run with --info or --debug option to get more log output. 

> Run with --scan to get full insights.   
* Get more help at https://help.gradle.org 
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.  
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. 
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings 
BUILD FAILED in 21s 
4 actionable tasks: 4 executed 
The command '/bin/sh -c /app/gradlew installDist' returned a non-zero code: 1 

As I was not able to find the proto file in the repo, so I have added the proto file. Is it the right way to add the file. Could you please share your pointers if this is the correct way of adding the proto file? Thanks in advance!

[LesnyRumcajs]

@odidev the proto file is located here. In the build script, one of those scenarios is transformed into proto/helloworld path. Please look at e.g. this benchmark which also uses Gradle.