toptal / crystalball

Regression Test Selection library for your RSpec test suite
https://toptal.github.io/crystalball/
MIT License
322 stars 41 forks source link

Crashes with Crystalball::MapStorage::NoFilesFoundError when following instructions #97

Closed ndbroadbent closed 6 years ago

ndbroadbent commented 6 years ago

rspec generates a file at ./execution_map.yml, but the crystalball command looks for tmp/execution_maps by default.

How can I keep these in sync?

ndbroadbent commented 6 years ago

Looking at the config, I can see that "map_path" and "execution_map_path" are actually 2 different things. I ran my test suite and crystalball generated a file at ./execution_map.yml, but I don't have any files at tmp/execution_maps.

ndbroadbent commented 6 years ago

I tried copying ./execution_map.yml to tmp/execution_maps, and it seemed to get a bit further. I have no idea if that's the correct thing to do, and the documentation is not very clear. But now it crashes with: Repository is not pristine! Please stash all your changes.

Is it not possible to run crystalball if you have unstaged changes? I don't want to have to commit all my changes before I run the tests.

ndbroadbent commented 6 years ago

OHHH I think the example crystalball.yml config is out of date: https://github.com/toptal/crystalball/blob/master/spec/fixtures/crystalball.yml

execution_map_path should be renamed to map_path.

ndbroadbent commented 6 years ago

I finally got something running! The main thing was wrapping the Crystalball config in a condition, so that it doesn't complain about changed files all the time:

if ENV['CRYSTALBALL']
  require 'crystalball'
  Crystalball::MapGenerator.start! do |config|
    config.map_storage_path = 'tmp/execution_map.yml'
    config.register Crystalball::MapGenerator::CoverageStrategy.new
  end
end

And I also fixed the map_storage_path in spec_helper, and set it in config/crystalball.yml:

map_path: 'tmp/execution_map.yml'

But I should point out that the documentation for examples_limit in the example config is not accurate:

# Maximum amount of examples which will be run automatically. Default: no limit.
# If prediction size is over the limit Crystalball will prune prediction to fit the limit.
examples_limit: 1

Crystalball doesn't prune the predictions to fit the limit, it just aborts all of the tests and doesn't run anything:

Example group size (14) is over the limit (1)
Aborting spec run
pluff commented 6 years ago

@ndbroadbent hi, thanks for feedback. You are probably using latest master which wasn't released yet. Anyway we'll address issues you pointed out pretty soon.

ndbroadbent commented 6 years ago

Oh, I see! That makes sense. Well I've managed to get it working, and it works very well! Thanks for building this!

pluff commented 6 years ago

https://github.com/toptal/crystalball/commit/e97ec75b4572fc9da7f198aa9e4581fd2d53a6c2 should fix this issue