opium in techempower frameworks benchmarks

[mudrz]

not really an issue, but just letting you know that opium was added to the techempower frameworks benchmarks https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/OCaml/opium

it didn't catch the last benchmarks run until a planned update of their tools https://github.com/TechEmpower/FrameworkBenchmarks/issues/6064

so the next benchmarks run will be delayed indefinitely

in the meantime please feel free to make changes and improvements

[tmattio]

Hi @mudrz,

That's awesome, thanks a lot for doing this!

I'm very curious to see how the next version of Opium (w/ Httpaf) will compare with the one you used for the benchmarks (w/ Cohttp). The benchmarks showing a better performance of Httpaf compared to Cohttp are at least 3 years old and Cohttp evolved a lot since then, so I wonder if they still stand.

[mudrz]

in the current implementation opium is pinned to a master commit with httpaf

https://github.com/TechEmpower/FrameworkBenchmarks/blob/21bb27fbbfb778e09cf048efd2e87af84c883097/frameworks/OCaml/opium/src/Makefile#L10

https://github.com/TechEmpower/FrameworkBenchmarks/blob/21bb27fbbfb778e09cf048efd2e87af84c883097/frameworks/OCaml/opium/opium.dockerfile#L15

but it's really easy to add a new dockerfile and use the cohttp version in order to compare

[mudrz]

edit: didn't notice that the httpaf version has a new API, so it's it would need more work to create a benchmark with cohttp;

also there are now results: https://www.techempower.com/benchmarks/#section=test&runid=bf3eceff-da94-46a8-87ab-6fc3ef39c12c&hw=ph&test=json&l=ziiku7-v&a=2&f=2-0-10-e8-74-9hc-6nq9ds-2-e8-hra0hs-f0g

@tmattio poster about the results here: https://discuss.ocaml.org/t/techempower-benchmarks-for-opium-running-httpaf-are-ready-how-to-improve-the-results/6627

There is something wrong with the plaintext results, not sure if it is in my implementation, or perhaps there's something in opium?

[tmattio]

Hi @mudrz,

There's definitely something going on with the plain text serialization. I'll look into it as soon as I have some spare time!

Thanks again for doing this, it's great to see where Opium positions itself in comparison with other frameworks 🙂

[mudrz]

the plaintext issue is related to httpaf and should be fixed: https://discuss.ocaml.org/t/techempower-benchmarks-for-opium-running-httpaf-are-ready-how-to-improve-the-results/6627/18?u=mudrz

(but there doesn't seem to be a new release)