Pass thru test-options opts to rspec to support running with Guard (for failure-exit-code)

rsanheim commented 2 years ago

For https://github.com/serpapi/turbo_tests/issues/14: Allow passing custom test options via the --test-options flag so that turbo_tests works with Guard. I believe guard passes thru the custom exit code to parallel_tests so it can properly capture failure exit codes and keep rerunning things and not abort...and turbo tests needs the same thing to work with guard.

This required tweaking how the runner reports exit statuses back to the CLI -- I wasn't super sure about that piece and how it will interact with the threads, but it is reporting the correct exit code for the limited specs I've added.

I chose not to support extra options via -- for a few reasons:

I wanted to get the custom_exit_code piece working in the simplest way possible to start to support guard usage 😁
When I attempted it initially, I was hitting issues with all sorts of edge cases where you may or may not have multiple -- in the command line, and then making sure you merge in things like tags. If I had more time I would refactor the arg parsing and test it much better in isolation (probably much like parallel-tests), and then work on getting that state to the runner in a nice, clean way.

Currently all specs are passing for me locally - I thought the pending specs were failing but that was user error, I had some errant whitespace added. ✅

I also refactored the specs a bit to be easier to read with more of the state / setup close to the actual thing being tested and asserted. Hope this all makes sense to you, let me know what you think @ilyazub.

rsanheim commented 2 years ago

Hey @ilyazub, I have this passing for test-options and ready for another review pass. Lemme know what you think!

john-h-k commented 1 year ago

Can I take over this work?

hartator commented 1 year ago

Can I take over this work?

Sure, PRs are always welcomed. 👍

rsanheim commented 1 year ago

Can I take over this work?

Please do, I haven't been able to carve out time to continue it and using turbo-test via guard was always a nice to have for me. I still use turbo-test every day for full build runs and it works great for that as-is.

Gloglok commented 1 year ago

Hey I'm very interested in this feature! Been using this gem on my own for a while and my team would love to use it too, but passes --test-options '--order rand' to parallel_tests.

I haven't contributed to open source before, do you think I could help finish this PR? 😄

ilyazub commented 1 year ago

@Gloglok sure, thank you!

Please also support passing test options after the end of options (--). Like in parallel_tests: https://github.com/grosser/parallel_tests/blob/7bebc55b7143afe09c4ad3d69c11fb0e15d44d77/lib/parallel_tests/cli.rb#LL310

What help is needed?

rsanheim commented 11 months ago

This is very out of date, and maybe has been handled upstream? In any case, closing this out.

serpapi / turbo_tests

Pass thru test-options opts to rspec to support running with Guard (for failure-exit-code) #20