rubytune / perf_check

PERRRFFF CHERRRRK!
17 stars 7 forks source link

[INFR-1205] add support for branch/master envars with tests #22

Closed aks closed 5 years ago

aks commented 7 years ago

This PR adds two new options: -e ENVAR=VALUE and -E ENVAR=VALUE, which can be provided multiple times to declare envars and values to be assigned for the test branch and the reference branch, respectively.

It also adds a -M option as a short-hand for --migrations.

This allows perf_check to test changes that depend on specific envar values.

There is a new example test to verify the use of the envar support at lines 52-74 in server_spec.rb.

All specs pass.

aks commented 7 years ago

The TravisCI failure is in the master branch -- which suggests that the test_app is missing the "railties" gem. Not sure how to fix the "master" branch?

can't find executable rails for gem railties. railties is not currently included in the bundle, 
perhaps you meant to add it to your Gemfile? (Gem::Exception)
sudara commented 7 years ago

A test case knows when its context is :reference https://github.com/rubytune/perf_check/blob/master/lib/perf_check/test_case.rb#L119

Maybe we can ask it here and send the appropriate env var into server.restart https://github.com/rubytune/perf_check/blob/master/lib/perf_check.rb#L115

aks commented 7 years ago

@siggymcfried: