ManageIQ / floe

Floe is a runner for Amazon States Language workflows
Apache License 2.0
0 stars 5 forks source link

[DEV] Improve performance of exe/floe specs #218

Closed kbrock closed 2 months ago

kbrock commented 2 months ago

Overview

The exe/floe tests take 3 seconds while the rest of the suite takes 0.8s

Is there a way to rewrite these 12 examples to not take so long?

Numbers

floe $ time bundle exec rake
Finished in 2.69 seconds (files took 0.12459 seconds to load)
403 examples, 0 failures

real    0m3.266s
user    0m2.156s
sys 0m0.800s

floe $ mv spec/exe/floe_spec.rb spec/exe/floe_spec-disabled.rb
floe $ time bundle exec rake
Finished in 0.26167 seconds (files took 0.1256 seconds to load)
391 examples, 0 failures

real    0m0.829s
user    0m0.592s
sys 0m0.179s
Fryguy commented 2 months ago

They have the slow tag on them, so from a local dev perspective, you can ignore them with rspec --tag ~slow

Fryguy commented 2 months ago

profiling wise - they each take about 0.18s

$ be rspec --tag slow --profile
Run options: include {:slow=>true}
............

Top 10 slowest examples (1.88 seconds, 84.3% of total time):
  exe/floe with a bare workflow and input
    0.19555 seconds ./spec/exe/floe_spec.rb:35
  exe/floe with a bare workflow and no input
    0.19535 seconds ./spec/exe/floe_spec.rb:27
  exe/floe with multiple bare workflows and --input
    0.1953 seconds ./spec/exe/floe_spec.rb:99
  exe/floe with a bare workflow and --input
    0.19005 seconds ./spec/exe/floe_spec.rb:43
  exe/floe with --workflow and --input
    0.18919 seconds ./spec/exe/floe_spec.rb:59
  exe/floe with mismatched workflow/input pairs
    0.18722 seconds ./spec/exe/floe_spec.rb:115
  exe/floe with multiple bare workflow/input pairs
    0.18386 seconds ./spec/exe/floe_spec.rb:83
  exe/floe displays help
    0.18268 seconds ./spec/exe/floe_spec.rb:8
  exe/floe with --workflow and no input
    0.18212 seconds ./spec/exe/floe_spec.rb:51
  exe/floe with no parameters
    0.17438 seconds ./spec/exe/floe_spec.rb:19

Finished in 2.22 seconds (files took 0.12381 seconds to load)
12 examples, 0 failures
Fryguy commented 2 months ago

I rewrote one of the tests to just call true to see if spawning overhead was slow, and it seems the absolute fastest we could run a test is ~0.01s, so there is room for improvement somewhere

  exe/floe displays help
    0.20902 seconds ./spec/exe/floe_spec.rb:20
  exe/floe runs `true`
    0.01494 seconds ./spec/exe/floe_spec.rb:8
Fryguy commented 2 months ago

It seems it takes about 0.13s to just require floe (I tried on Ruby 3.1, 3.2, and 3.3). That feels really slow 🤷

$ time ruby -I lib -r floe -e 'puts Floe::VERSION'
0.11.2
ruby -I lib -r floe -e 'puts Floe::VERSION'  0.08s user 0.04s system 86% cpu 0.138 total
Fryguy commented 2 months ago

Over half of that 0.13s is just regular Ruby startup timing:

$ time ruby -e ''
ruby -e ''  0.05s user 0.03s system 79% cpu 0.098 total