RefugeRestrooms / refugerestrooms

REFUGE restrooms indexes and maps safe restroom locations for trans, intersex, and gender nonconforming individuals.
http://www.refugerestrooms.org
GNU Affero General Public License v3.0
891 stars 263 forks source link

Tests aren't passing (except in an old, end-of-life Ubuntu release under Travis CI) #601

Closed DeeDeeG closed 4 years ago

DeeDeeG commented 4 years ago

Problem

Tests are failing anywhere but Travis CI's old Ubuntu 14.04 "Trusty" base image.

See this recent test run for an example: https://travis-ci.com/RefugeRestrooms/refugerestrooms/builds/130835299#L956-L981

Full error (click, or give keyboard focus and press enter, to expand) ```ruby An error occurred while loading ./spec/api/v1/restrooms_spec.rb. Failure/Error: config.include Locations NameError: uninitialized constant Locations # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:79:in `block in load_missing_constant' # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:8:in `without_bootsnap_cache' # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:79:in `rescue in load_missing_constant' # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:58:in `load_missing_constant' # ./spec/support/rspec.rb:19:in `block in ' # ./spec/support/rspec.rb:18:in `' # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require' # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `block in require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register' # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require' # ./spec/spec_helper.rb:17:in `block in ' # ./spec/spec_helper.rb:17:in `each' # ./spec/spec_helper.rb:17:in `' # ./spec/api/v1/restrooms_spec.rb:1:in `require' # ./spec/api/v1/restrooms_spec.rb:1:in `' # ------------------ # --- Caused by: --- # NameError: # uninitialized constant Locations # /usr/local/bundle/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:60:in `block in load_missing_constant' ```

.

There are eight of these errors, one for each inclusion of Locations (which I believe is intended to be a "module" in Ruby terminology).

Scope / difficulty

Looks to me like a Ruby syntax error. Anyone knowledgeable about re-using code from one file to another in Ruby should be able to fix this, at least in theory.

Impact

Allows us to keep running our CI tests, but on a new Travis CI base image (Ubuntu 16.04 "Xenial" or newer).

Might also make running tests locally work. As far as I know, the tests don't pass anywhere but Travis CI's Ubuntu 14.04 "Trusty" environment. I would love to be able to run these tests natively on my local machine, or locally in a Docker container.

The current image, Ubuntu 14.04 "Trusty," is already in its end-of-life period. It is no longer supported. Travis CI is likely to keep supporting it for a while, due to how many projects are probably still using it. In theory, though, they could decide to stop offering Trusty CI runs at any time if they wanted to.

Fixing this adds certainty that we can keep testing on Travis CI.

Proposal

Change the code for our tests to make them pass everywhere, not just in this older Travis CI test environment.

How to actually do this:

Probably research the proper way to include modules, and edit the Ruby code in the spec/ directory of this project.

DeeDeeG commented 4 years ago

Here are my comments from Slack, since they're relevant and I think they should be available for anyone trying to solve this issue:

We have a weird issue with the tests on Travis CI. It's not [happening due to any recent change in] our code, since I see it when re-running the CI build on commits that have already passed. Basically, one of the test support files is trying to include the Locations module from another file, but it instead sees Locations as an undefined constant. I have seen this come up before, and I am puzzled by it every time. Any insights would be appreciated. If anyone feels like sleuthing through some RSpec bugs... It could help develop branch's CI stay passing more reliably. https://travis-ci.com/RefugeRestrooms/refugerestrooms/builds/130835299


(Referring to the website's CI, as opposed to CI for either of the mobile apps.)


I'm pretty sure the Travis CI issue happened when our builds switched to their new Ubuntu "Xenial" 16.04 images, versus their Ubuntu "Trusty" 14.04 images. Hopefully I can figure out the precise difference between the images that is causing this problem.


(Worst comes to worst, we can just specify dist: trusty in our .travis.yml)


I can confirm tests pass in Travis CI when dist is set to trusty... But weirdly, I have trouble reproducing passing tests anywhere other than Travis CI trusty. I can't reproduce locally on my Linux machine, in a Virtualbox install of Ubuntu Trusty, with a Vagrant install of Ubuntu Bionic, Travis xenial... It seems like Travis CI trusty is the odd case out, and our tests don't really work anywhere else. If anyone can confirm, or tries and gets different results, let me know. I'd be very curious to get to the bottom of this. I am beginning to think the tests are the problem, not Travis CI.


I'm planning to switch Travis's dist setting to trusty for now, just to know if PRs still pass. I'm not good enough with Ruby at the moment, specifically including code from across separate files, to fix this issue for the general case. (I have no idea why this works in Travis trusty, since it doesn't appear work anywhere else, as far as I can tell.)

DeeDeeG commented 4 years ago

Setting Travis CI's dist setting to bionic doesn't work either.

stardust66 commented 4 years ago

I'm looking into this. Going to make a pull request to run tests with Travis, because for some reason that error doesn't pop up in Docker on my machine.

Edit: Never mind it's failing on my machine as well.

DeeDeeG commented 4 years ago

Thank you @stardust66!