Flaky spec: Emails Budgets Selected investment

[bertocq]

What

Tests that fail randomly are called "flakies", this one seems to be one:

Randomized seed: 19467

Travis failed build: https://travis-ci.org/consul/consul/jobs/337981250

Failure:

    1) Emails Budgets Selected investment
     Failure/Error: expect(email).to have_subject("Your investment project '#{investment.code}' has been selected")
       expected the subject to be "Your investment project '2018-26' has been selected" but was "Your investment project '2018-25' has been selected"
     # ./spec/features/emails_spec.rb:431:in `block (3 levels) in <top (required)>'

How

• [ ] Explain why the test is flaky, or under which conditions/scenario it fails randomly
• [ ] Explain why your PR fixes it

Tips for flaky hunting

Random values issues

If the problem comes from randomly generated values, running multiple times a single spec could help you reproduce the failure by running at your command line:

for run in {1..10}
do
  bin/rspec ./spec/features/budgets/investments_spec.rb:256
done

You can also try running a single spec in Travis: Add option :focus to the spec and push your branch to Github, for example:

scenario 'Show', :focus do

But remember to remove that :focus changes afterwards when submitting your PR changes!

Test order issues

Running specs in the order they failed may discover that the problem is that a previous test sets an state in the test environment that makes our flaky fail/pass. Tests should be independent from the rest.

After executing rspec you can see the seed used, add it as an option to rspec, for example: bin/rspec --seed 55638 (check Randomized seed value at beginning of issue)

Other things to watch for

• Time related issues (current time, two time or date comparisons with miliseconds/time when its not needed)
• https://semaphoreci.com/community/tutorials/how-to-deal-with-and-eliminate-flaky-tests

[MariaCheca]

Closed via https://github.com/consul/consul/pull/2695