boc-tothefuture / openhab-jruby

JRuby Libraries for Openhab
Eclipse Public License 2.0
6 stars 9 forks source link

bring in rspec-openhab-scripting gem #639

Closed ccutrer closed 1 year ago

ccutrer commented 1 year ago

and convert (most) _type.feature tests to rspec in-process tests.

By running tests in-line, and in-process, massive speed increases are possible, as well as ease of writing tests. You don't have to wrap everything in a rules file, that then gets deployed, wait for OpenHAB to detect it, and then check the results by continually checking the log file. This also allows more particular testing (like the rspec be matcher in the PercentType spec that also checks for type equality).

For this initial proof-of-concept, I converted 28+ cucumber scenarios to specs. The runtime on my computer:

cody@opennuc:~/src/openhab-jruby$ time rspec

Randomized with seed 8442
............................

Finished in 18.4 seconds (files took 2.48 seconds to load)
28 examples, 0 failures

Randomized with seed 8442

real    0m26.233s
user    1m46.645s
sys 0m2.334s

The vast majority of that time is booting up - the actual specs (all of them, not each one) took a fraction of a second to run. This is in stark contrast to cucumber's infrastructure that takes 30-60s to start openhab, and then a minimum of 5 seconds per scenario.

ccutrer commented 1 year ago

@jimtng @boc-tothefuture : this is ready for review now!

ccutrer commented 1 year ago

Note that with this commit, you can also do bundle console, and get a JRuby REPL inside an OpenHAB environment. It's amazing for exploring how things work when you're implementing new features/fixing bugs.

ccutrer commented 1 year ago

I had a look at the test run and noticed that there's a lack of details in the test output. I suppose failures would show up there. I've just gotten used to seeing all the details from cucumber.

The output is configurable. The default is when running the entire suite (just rspec), use the dots, with a summary at the end (and full exception info for failures), or when running targeted (rspec spec/decimal_type_spec.rb) it lists each individual spec by name as it runs, with the same summary. There are lots more output options as well.

Lastly, we'd obviously lose the pickles pages for the tests that are no longer done in cucumber, such as https://boc-tothefuture.github.io/openhab-jruby/pickles/decimal_type.html I don't usually look at the pickles output and I doubt anyone does.

A couple things here:

ccutrer commented 1 year ago

Oops. Accidentally pushed to the wrong repo, "merging this". Need to undo.