Automatically handle parallel execution

[andrewhampton]

First, this gem is off to a great start! Thank you for putting your time into an open source library that helps the ruby community move forward!

I'm currently using parallel_tests for a test suite that takes a couple hours to run without parallelization. Two killer features it has are:

• setup script: bin/rails parallel:setup
• flag: -n [PROCESSES] How many processes to use, default: available CPUs

If you're not already familiar with these, the main idea here is parallel_tests expects you to run multiple processes on the same machine. Each process will get an env var set, TEST_ENV_NUMBER, which gives each process a distinct ID to use. This ID can be used in the Rails database.yml like database: test_<%= test_env_number %>, which allows each process to use the same DB server, but unique databases during the test run.

Are there plans to for rspecq to do something similar?

I think the main benefit of handling this in rpsecq is it solves the problem once, rather than requiring all consumers to do similar setup work. Of course, that setup work isn't necessary if your runners are distributed. But this would give parallelt_test users a convenient on ramp when switching to rspecq.

[agis]

Thanks for the kind words @andrewhampton! I'm open to incorporating a separate executable that would take care the job of spinning up multiple rspecq workers in the same host.

However, note that we don't currently use rspecq this way, instead we rely on containers each having its own set of services (MySQL, MongoDB, Elasticsearch etc.). Which means this executable naturally wouldn't be as supported as rspecq itself. On the other hand, I guess it'd be minimal so maybe it wouldn't need so much support. :)

[agis]

Would something like that help with your use case?

[andrewhampton]

For sure! It's not too hard, but implementing it in the gem gives everyone a canonical implementation to all use.

On Wed, Sep 2, 2020, 8:21 AM Agis Anastasopoulos notifications@github.com wrote:

Would something like that help with your use case?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/skroutz/rspecq/issues/28#issuecomment-685696695, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABNVNJNJRGUAKZNV6ETH6LSDY2GFANCNFSM4QORT7LA .