Open RobinDaugherty opened 3 years ago
That's a fair point. I cannot think in multiple ways to easily validate that, one idea I just had was:
The rake would load the Rails env so if the gem was corrupted in any way the rake would not execute and would fail. I'm just not 💯 sure on how to make the step 1 to work, but I would be happy to give it a try. What do you think?
I think this might be a very simple thing actually. I have various Rails and Rack apps I've built to manually test in. One or more of these could be downloaded in our CI, then:
This could be done using a build stage in Travis, or maybe just tacked onto the existing CI run, running on each combination of Ruby/Rails/binding_of_caller/haml.
480 and #481 fixed an issue that I introduced by referring to a constant that wasn't loaded yet.
CI didn't fail, even though every app failed the moment it tried to load.
I think the solution here is to add a smoke test that runs an app of some kind (Rails/Sinatra/pure Rack) and make sure the app starts. This actually might be more effective than all of the specs we have, which currently have to cover a lot of different cases (with Rails, each version of rails, etc).