undefined method `each_pair' for nil:NilClass with gitlab rails application

[harshanails]

Describe the bug

rake coverband:coverage --trace
** Invoke coverband:coverage (first_time)
** Execute coverband:coverage
rake aborted!
NoMethodError: undefined method `each_pair' for nil:NilClass
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/coverband-5.2.5/lib/coverband/reporters/console_report.rb:11:in `report'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/coverband-5.2.5/lib/coverband/utils/tasks.rb:17:in `block (2 levels) in <top (required)>'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/sentry-ruby-core-5.1.1/lib/sentry/rake.rb:24:in `execute'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/bin/rake:25:in `load'
/Users/harshamuralidhar/.asdf/installs/ruby/3.0.5/bin/rake:25:in `<main>'

To Reproduce Steps to reproduce the behavior:

1. Make sure rake -T coverband --trace command is working and make sure COVERBAND_REDIS_URL is set.
2. Make sure route /coverage is defined in routes.rb.
3. Run rake coverband:coverage

Expected behavior Coverage should be reported in the path /coverage. Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• MacOS Ventura 13.1

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context I am using an external redis server, not the one used by gitlab app.

[harshanails]

Oh, when i opened the application and performed some actions on the app, the command rake coverband:coverage worked.

[harshanails]

Though I am seeing undefined method instance_observe_duration for nil:NilClass in coverage path.

[harshanails]

Oh, when i opened the application and performed some actions on the app, the command rake coverband:coverage worked.

I can confirm that when no test/action is performed on the rails application before running rake coverband:coverage, it throws the undefined method each_pair for nil:NilClass error. On running, rake coverband:clear I was able to reproduce the behaviour.

[harshanails]

Although I am still having trouble mounting the coverage path in the rails app, I will close this issue.

[harshanails]

I can confirm that when no test/action is performed on the rails application before running rake coverband:coverage, it throws the undefined method each_pair for nil:NilClass error. On running, rake coverband:clear I was able to reproduce the behaviour.

Actually I am back to square one. Even after performing actions on the rails app it throws this error :(

[danmayer]

Sorry, I haven't looked at this, I need some time to pull down gitlab rails and to better understand what exactly is going on

[harshanails]

I tried this again, but just mounting the coverage route in config/routes.rb of the Gitlab rails app and adding coverband to the Gemfile. Also started a redis-server locally on localhost:6379. Seeing the following error in Gitlab rails log:

Class> undefined method `instance_observe_duration' for nil:NilClass
2024-04-18_03:09:41.90959 rails-web             : E, [2024-04-18T13:09:41.909495 #57889] ERROR -- : Coverband: view_tracker failed to store, error NoMethodError
2024-04-18_03:09:47.44703 rails-web             : E, [2024-04-18T13:09:47.446805 #57887] ERROR -- : coverage failed to store
2024-04-18_03:09:47.44714 rails-web             : E, [2024-04-18T13:09:47.447015 #57887] ERROR -- : Coverband Error: #<NoMethodError: undefined method `instance_observe_duration' for nil:NilClass> undefined method `instance_observe_duration' for nil:NilClass
2024-04-18_03:09:47.44728 rails-web             : E, [2024-04-18T13:09:47.447180 #57887] ERROR -- : Coverband: view_tracker failed to store, error NoMethodError

[harshanails]

It could be because the local setup of the Gitlab application also uses a redis server.

[harshanails]

When I shutdown the redis-server on localhost:6379, the error is:

2024-04-18_04:40:04.69467 rails-web             : E, [2024-04-18T14:40:04.694487 #89350] ERROR -- : coverage failed to store
2024-04-18_04:40:04.69480 rails-web             : E, [2024-04-18T14:40:04.694659 #89350] ERROR -- : Coverband Error: #<Redis::CannotConnectError: Connection refused - connect(2) for [::1]:6379 (redis://localhost:6379)> Connection refused - connect(2) for [::1]:6379 (redis://localhost:6379)
2024-04-18_04:40:04.69638 rails-web             : E, [2024-04-18T14:40:04.696243 #89350] ERROR -- : Coverband: Coverband::Collectors::ViewTracker failed to store, error Redis::CannotConnectError info Connection refused - connect(2) for [::1]:6379 (redis://localhost:6379)
2024-04-18_04:40:27.19944 rails-web             : Since there is no EDITOR or BETTER_ERRORS_EDITOR environment variable, using Textmate by default.
2024-04-18_04:40:27.21010 rails-web             : E, [2024-04-18T14:40:27.209914 #89349] ERROR -- : coverage failed to store
2024-04-18_04:40:27.21110 rails-web             : E, [2024-04-18T14:40:27.210952 #89349] ERROR -- : Coverband Error: #<Redis::CannotConnectError: Connection refused - connect(2) for [::1]:6379 (redis://localhost:6379)> Connection refused - connect(2) for [::1]:6379 (redis://localhost:6379)
2024-04-18_04:40:27.21733 rails-web             : E, [2024-04-18T14:40:27.217188 #89349] ERROR -- : Coverband: Coverband::Collectors::ViewTracker failed to store, error Redis::CannotConnectError info Connection refused - connect(2) for [::1]:6379 (redis://localhost:6379)

[danmayer]

so is this just pulling down gitlab and then running it on the main branch and then trying to add coverband? If so I can try to reproduce that... I have never used git lab so I don't know much about it.

[danmayer]

is this an open source code repository I can try to get a copy of to run and fix it or is this a closed source repo...

[danmayer]

If it is this repo I can give it a shot https://github.com/gitlabhq/gitlabhq.git

[danmayer]

oof getting gitlab running is quite the task it is a huge install... but I will give it a go

[harshanails]

Update: I was able to get it working with the Gitlab rails app with this config: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/149950/diffs?pin=e2b741f5feac38042b9bcc53faa2abe9f6dafaae#e2b741f5feac38042b9bcc53faa2abe9f6dafaae_0_6

  config.store = Coverband::Adapters::RedisStore.new(::Redis.new(url: 'redis://localhost:6379',
    custom: { instrumentation_class: 'SharedState' }))

Thanks for keeping tabs on the issue.

[danmayer]

nice @harshanails that is amazing. Let me know how it works for you all or if you need help tweaking any configuration values.

I generally recommend setting a longer background_reporting_sleep_seconds but if this is just on a test server vs a larger cluster it should be fine... If you are running on a bunch of machines tweaking that and some other options will help avoid overloading your redis.

Thanks for spending time to figure it all out. I am looking forward to hearing if it works well for you all.

[danmayer]

OK, since this issue is fixed I am going to close it but feel free to keep adding comments or updates... or open a new issue if you have any problems or questions.

[harshanails]

OK, since this issue is fixed I am going to close it but feel free to keep adding comments or updates... or open a new issue if you have any problems or questions.

Thanks @danmayer!

It is working well for our use case, but I have a question regarding config.background_reporting_sleep_seconds configuration. Is this config for "how often coverband persists coverage data to redis cache?"

And does ::Coverband.configuration.store.coverage force coverband to write data to redis? Or does it retrieve the coverage already stored in redis?

[danmayer]

yes config.background_reporting_sleep_seconds is how long it waits between sending reports to redis. So having that longer will put less load on your redis but will mean your coverage is less up to date. If you are running many servers increasing that along with the reporting wiggle will help avoid to many reports hitting redis at the same time. The coverage is aggregated in memory between each report.

Note that it reports per process so how many workers are buidling reports and sending to redis depends on your webserver and configurations for example using Puma with threads will have a shared reporter per puma process.

[harshanails]

yes config.background_reporting_sleep_seconds is how long it waits between sending reports to redis. So having that longer will put less load on your redis but will mean your coverage is less up to date. If you are running many servers increasing that along with the reporting wiggle will help avoid to many reports hitting redis at the same time. The coverage is aggregated in memory between each report.

Note that it reports per process so how many workers are buidling reports and sending to redis depends on your webserver and configurations for example using Puma with threads will have a shared reporter per puma process.

@danmayer Thank you, that was a helpful response. Another question I had was - I have require 'false' in the Gemfile for coverband because for our use case, we don't want it to be part of the production application.

So question is basically - does the background reporting start as soon as require 'coverband' is called in the code? And will restarting the rails application stop the background reporting? Is there a way to stop the background reporting by calling a function perhaps?