RSpec is showing issues

[DiemBTran]

RSpec is has issues loading FactoryBot

Details

An error occurred while loading rails_helper. Failure/Error: config.include FactoryBot::Syntax::Methods NameError: uninitialized constant FactoryBot # ./spec/support/factory_bot.rb:4:in `block in ' # ./spec/support/factory_bot.rb:3:in `' # ./spec/rails_helper.rb:42:in `block in ' # ./spec/rails_helper.rb:42:in `each' # ./spec/rails_helper.rb:42:in `' Run options: include {:focus=>true} exclude {:aws=>true} All examples were filtered out; ignoring {:focus=>true} All examples were filtered out Top 0 slowest examples (0 seconds, 0.0% of total time): Finished in 0.00006 seconds (files took 24.53 seconds to load) 0 examples, 0 failures, 1 error occurred outside of examples Coverage report generated for RSpec to /app/samvera/hyrax-webapp/coverage. 1025 / 6793 LOC (15.09%) covered. Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected

Workaround

Running "bundle exec rake spec" is our current workaround.

Rspec displays issues with the the themes, label, and generic works (previously mentioned).

Details

Finished in 12 minutes 50 seconds (files took 24.21 seconds to load) 642 examples, 14 failures, 56 pending Failed examples: rspec ./spec/features/create_generic_work_spec.rb:42 # Create a GenericWork a logged in user is expected to have text "Add folder" rspec ./spec/features/labels_spec.rb:12 # Site labels configuration as an administrator application name updates the application name in the brand bar rspec ./spec/features/cultural_repository_theme_spec.rb:21 # Admin can select cultural repository theme as a repository admin sets the cultural repository theme when the theme form is saved rspec ./spec/features/cultural_repository_theme_spec.rb:38 # Admin can select cultural repository theme when the cultural repository theme is selected renders the partials in the theme folder rspec ./spec/features/show_page_theme_spec.rb:42 # Admin can select show page theme as a repository admin sets the themes when the theme form is saved rspec ./spec/features/show_page_theme_spec.rb:27 # Admin can select show page theme as a repository admin has a select box for the show page themes rspec ./spec/features/show_page_theme_spec.rb:59 # Admin can select show page theme when a show page theme is selected renders theme notes and wireframe rspec ./spec/features/show_page_theme_spec.rb:70 # Admin can select show page theme when a show page theme is selected renders the partials in the theme folder rspec ./spec/features/show_page_theme_spec.rb:85 # Admin can select show page theme when a show page theme is selected updates the show theme when the theme is changed rspec ./spec/features/appearance_theme_spec.rb:46 # Admin can select home page theme as a repository admin sets the themes when the theme form is saved rspec ./spec/features/appearance_theme_spec.rb:29 # Admin can select home page theme as a repository admin has a select box for the home, show, and search pages themes rspec ./spec/features/appearance_theme_spec.rb:119 # Admin can select home page theme when a home page theme is selected renders theme notes and wireframe rspec ./spec/features/appearance_theme_spec.rb:67 # Admin can select home page theme when a search results theme is selected updates the search results page with the selected layout view rspec ./spec/features/institutional_repository_theme_spec.rb:18 # Admin can select institutional repository theme as a repository admin sets the institutional repository theme when the theme form is saved Randomized with seed 43931

[DiemBTran]

Shana Oct 2022

---

@rtkuehn We may need to pair on this one. We are not seeing the issues you are having. That uninitialized constant usually means something is not being loaded properly or a gem (FactoryBot) isn't installed.

You can see that we run the spec suite each time we push code to gitlab, in our pipelines section. It gets configured in .gitlab-ci.yml

These days, if it fails it's because of a flaky feature spec; normally there is only 1 failure. Flaky/flappy means that sometimes this spec passes and sometimes it doesn't. It usually has to do with a timing issue; the headless browser is looking for an element on the page that hasn't loaded yet, so it marks it as a fail (noted by an exclamation point that you can click into, to see the results)

We are aware of this but haven't poked at it since we've been prioritizing feature requests and bug fixes (and because it's a flake). Oftentimes, when we notice the failure on the pipeline, re-running it is enough to make it pass (noted by the green check marks if the entire suite passes).

All that is to say, when you run the specs locally it should be similar to the results found when running specs in the pipeline. But yours seem to be drastically different so I wonder if something else is going on.

What command are you using to run rspec? RAILS_ENV=test bundle exec rspec spec/ inside of the docker web bash container?

[DiemBTran]

Randy Oct 2022

---

@ShanaLMoore I made a few updates to this post today.

Yeah, I have no idea why these are failing.

What I do know... running rspec or bundle exec rspec spec/ rather than bundle exec rake spec in the web bash container will result in trivial failures (e.g. uninitialized constant FactoryBot). I can fix the FactoryBot error by requiring factory_bot in spec/support/factory_bot.rb, but then another uninitialized constant error will pop up...so I decided to just leave this be. bundle exec rake spec works for me! :)

That aside... bundle exec rake spec produces some weird errors that I have yet to be able to reproduce if I walk through the spec steps manually in the app. For instance, changing the label name in the app modifies the application name in the html...and the generic work errors (I've seen a couple different ones) don't seem to exist when manually walking through creating a generic work. The theming error are more difficult because I believe our custom css is overriding most of those. :\

So, I'm stumped. I'm kinda hoping these are all just trivial and we can close this and call it done...but I reckon I'm just looking for confirmation that I'm not overlooking something critical.

/cc @rachel.howard

[DiemBTran]

Shana Oct 2022

---

Pairing Notes (randy is remoted in)

1. unset RAILS_ENV got rid of the factroybot unitialized constant error. You could also do RAILS_ENV=test bundle exec rspec spec to avoid this error. It thought it was in dev mode which doesn't load rspec/factorybot. Rob's recommendation is to remove it entirely from the .env file.
2. Team Sprint is moving this repo to Github. This will allow tests to run in Github and we'll see if the LV repo passes there. Possible config issues between mac and linux could be the cause. We could also share the circle ci templates with Randy at that time, for him to enable on his version of the repo.
3. LV may want to consider a maintenance contract. Kevin emailed the client.

[DiemBTran]

Shana Oct 2022

---

UPDATE: there are differences between their repo (main) and ours (ss-changes/main). The client has made changes to the css and html. Knowing this, I believe these failures may be valid after all.

I checked out Randy's code directly and ran the specs. I am seeing quite a few failures as well. However, because our code is not the cause of the failures, we've decided to hold off on fixing these specs for now.

The client (Randy) is ok with taking note of the errors for future references and understands that fixing this may be out of scope with our current contract.

cc @rtkuehn @orangewolf @rachel.howard

[DiemBTran]

Shana Oct 2022

---

I've shared with Randy that any time css and html files get changed, they will need to update the specs. Because I assume these changes are intentional, if the client decides to fix the specs themselves they will likely need to update the specs to reflect the changes of the code; the specs accurately no longer reflect what the users sees.

If the client does not want to address these now, instead of deleting specs that are no longer relevant, they can skip them so that they aren't being reported as failures anymore. I'd recommend leaving a comment explaining the reason for the skip. These specs will then be reported as pending vs failures, and maybe they can be addressed by us at a later time (if they proceed with a maintenance contract).