How to run benchmarks? - Githubissues

elixir-grpc / grpc

An Elixir implementation of gRPC

https://hex.pm/packages/grpc

Apache License 2.0

1.38k stars 212 forks source link

How to run benchmarks? #268

Open DaniruKun opened 2 years ago

DaniruKun commented 2 years ago

Describe the question I have tried following the exact instructions in the benchmarks subdirectory, but running the tests always fails. Could you please provide up-to-date instructions on how to setup an environment for benchmarks?

Versions:

OS: macOS 12.5 21G72 x86_64
Erlang:Erlang/OTP 25 [erts-13.0.3] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns]

Elixir 1.13.3 (compiled with Erlang/OTP 25)

Elixir:
mix.lock(grpc, gun, cowboy, cowlib):

polvalente commented 2 years ago

I think we're gonna need @tony612's input here. The fork is based on a really outdated version of grpc, and what fails (at least for me) is some part of the build process. There aren't many additions, but we need a commit updating the fork.

Additions in question: https://github.com/grpc/grpc/compare/master...tony612:grpc:elixir-bench

Alternatively, we could find a way to call the benchmark as a library instead and have this custom code as part of this repo.

tony612 commented 2 years ago

grpc's benchmark is complex and it may change as time goes by, so it's not strange to see the failing. I think we need to revisit the benchmarks and lots of code may need to be changed. It will be better if we can find a way to not fork grpc/grpc.