search

sharkdp / insect

High precision scientific calculator with support for physical units
https://numbat.dev/
MIT License
3.18k stars 126 forks source link

Explore using purescript-backend-optimizer to improve JavaScript output #335

Closed triallax closed 1 year ago

triallax commented 2 years ago

Today I saw purescript-backend-optimizer's announcement on Discourse, and it features a bunch of great optimizations such as inlining, and claims to substantially improve the size and speed of the produced JavaScript.

sharkdp commented 2 years ago

That sounds really cool. Thank you for the reference.

triallax commented 1 year ago

I tried this locally with purescript-backend-optimizer 1.3.2 and the current master, and the results aren't as overwhelming as I hoped. It decreases the size of the minified and bundled JS file a little (but the difference is not very large). As for the benchmarks, here are my results (YMMV):

master:

Command Mean [ms] Min [ms] Max [ms] Relative
Startup time (insect '1+1') 179.0 ± 1.8 176.6 181.3 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
Startup time, stdin mode (insect < computations-1.ins) 206.2 ± 8.1 200.2 231.4 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
Evaluation speed (insect < computations-160.ins) 1083.9 ± 33.4 1048.1 1160.0 1.00

With purescript-backend-optimizer 1.3.2:

Command Mean [ms] Min [ms] Max [ms] Relative
Startup time (insect '1+1') 183.1 ± 1.0 181.6 185.5 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
Startup time, stdin mode (insect < computations-1.ins) 212.8 ± 3.3 208.7 221.9 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
Evaluation speed (insect < computations-160.ins) 855.5 ± 66.8 810.8 1034.1 1.00

As you can see, the startup time does not seem to be significantly affected, but using purescript-backend-optimizer did provide a nice increase in the evaluation speed benchmark.

The main question is whether this small but significant gain in evaluation speed (and the even smaller decrease in bundle size) is worth it. purescript-backend-optimizer is still not very mature, and code generation bugs that may impact calculations have been reported in the past (e.g. https://github.com/aristanetworks/purescript-backend-optimizer/issues/38, though admittedly this probably wouldn't have a huge impact). I think the benefits don't outweigh the possibility of causing bugs as of now (though we may want to revisit it in the future when it's become more mature and generates more performant code).

sharkdp commented 1 year ago

Thank you so much for looking into this.

The main question is whether this small but significant gain in evaluation speed (and the even smaller decrease in bundle size) is worth it. purescript-backend-optimizer is still not very mature, and code generation bugs that may impact calculations have been reported in the past (e.g. aristanetworks/purescript-backend-optimizer#38, though admittedly this probably wouldn't have a huge impact). I think the benefits don't outweigh the possibility of causing bugs as of now (though we may want to revisit it in the future when it's become more mature and generates more performant code).

I fully agree. Another reason to keep things as they are is the added toolchain/CI complexity, if we were to add purescript-backend-optimizer.

sharkdp commented 1 year ago

I'd like to close this, given that we now have Numbat which is orders of magnitude faster: https://github.com/sharkdp/numbat