Top level functions slower than module functions in Elixir <1.16

[sabiwara]

Hello,

Thanks a lot for this amazing project, I have been using it intensively 💜

Reporting this mostly for the record, since it was actually an Elixir issue but has been fixed in elixir 1.16.0-rc.1. But maybe it can be documented to increase its visibility?

You can see a reproduction here, confirming both the issue (top level function 1.5x slower than module function on 1.15) and the fix (same speed on latest rc).

The discrepancy was due to disabled compiler optimizations (no_bool_opt, no_ssa_opt), which were meant to speed up the execution of meta-programming but are actually only needed for the module body.

[PragTob]

@sabiwara :scream:

Hey, thanks for the report! Do you know how far back this issue goes? It was fixed in 1.16.0-rc.1 but since when did it exist? We had been discussing recommending people to only use it from modules.

Also, I'd guess the issue only appears in "full" top level mode. i.e. if you put all of Benchee.run into a module function it likely won't occur?

[PragTob]

• I can't find it in the elixir changelog itself, for visibility might be good to add it there as well: https://github.com/elixir-lang/elixir/blob/v1.16/CHANGELOG.md

[sabiwara]

It was fixed in 1.16.0-rc.1 but since when did it exist?

This has been introduced with this commit and has been there since 1.14 as far as I can tell. 1.13.4 doesn't have the optimization removal.

Also, I'd guess the issue only appears in "full" top level mode. i.e. if you put all of Benchee.run into a module function it likely won't occur?

That is correct, I just confirmed it in practice.

I can't find it in the elixir changelog itself, for visibility might be good to add it there as well: https://github.com/elixir-lang/elixir/blob/v1.16/CHANGELOG.md

I agree 👍 Maybe we could announce it on the elixir forum in the specific context of Benchee as well.

[PragTob]

Alright, I also went ahead copied your benchmark and ran it on the different versions: https://github.com/PragTob/elixir_playground/blob/main/bench/top_level_functions.exs

I'll PR the changelog (if you haven't already) to check my understanding. Already reached out to an elixiforum mod about how to best post about it.

Will probably also find some section in the README for it and mention it in the changelog for a version I'll hopefully release some time this week but I've been too optimistic in the past :joy:

[sabiwara]

I'll PR the changelog (if you haven't already) to check my understanding. Already reached out to an elixiforum mod about how to best post about it.

I haven't yet, please go ahead 🙂

[PragTob]

PR filed :point_up:

[PragTob]

407 filed, I'll still fix up a bit and after it's merged probably put the same content with a bit more color into a blog post to PSA folks