search

renovatebot / renovate

Home of the Renovate CLI: Cross-platform Dependency Automation by Mend.io
https://mend.io/renovate
GNU Affero General Public License v3.0
17.27k stars 2.26k forks source link

Bundler failing with `ERROR: Undefined sourceLine` #5232

Closed bilby91 closed 4 years ago

bilby91 commented 4 years ago

What Renovate type are you using?

Renovate: Self hosted(renovate/renovate:19.109.0) Platform: GitLab

Describe the bug

When running renovate on a ruby project, the given error is raised:

ERROR: Undefined sourceLine (repository=org/my-repo)

Did you see anything helpful in debug logs?

After the above error is raised, the following errors show up on gem update attempts:

          "stdout": "ruby 2.6.4p104 (2019-08-28 revision 67798) [x86_64-linux]\nSuccessfully installed bundler-1.17.2\n1 gem installed\nCould not locate Gemfile\n",
          "stderr": "Unable to find image 'renovate/ruby:2.6.4' locally\n2.6.4: Pulling from renovate/ruby\n2746a4a261c9: Pulling fs layer\n4c1d20cdee96: Pulling fs layer\n0d3160e1d0de: Pulling fs layer\nc8e37668deea: Pulling fs layer\n0f17882f5071: Pulling fs layer\nab3f8384cfc6: Pulling fs layer\n9b1ce390f4af: Pulling fs layer\n69577f69924e: Pulling fs layer\n21281402473f: Pulling fs layer\n35f0abeb72e2: Pulling fs layer\nb04aed4ed1be: Pulling fs layer\nba9a469782b4: Pulling fs layer\nfbc0d5cbe920: Pulling fs layer\nb7f6a1ab3dd4: Pulling fs layer\n6e84e72c3ba3: Pulling fs layer\n3baafffdaf65: Pulling fs layer\n21281402473f: Waiting\n35f0abeb72e2: Waiting\nb04aed4ed1be: Waiting\nba9a469782b4: Waiting\nfbc0d5cbe920: Waiting\nb7f6a1ab3dd4: Waiting\n6e84e72c3ba3: Waiting\n3baafffdaf65: Waiting\nc8e37668deea: Waiting\n0f17882f5071: Waiting\nab3f8384cfc6: Waiting\n9b1ce390f4af: Waiting\n69577f69924e: Waiting\n0d3160e1d0de: Verifying Checksum\n0d3160e1d0de: Download complete\nc8e37668deea: Verifying Checksum\nc8e37668deea: Download complete\n0f17882f5071: Verifying Checksum\n0f17882f5071: Download complete\nab3f8384cfc6: Verifying Checksum\nab3f8384cfc6: Download complete\n2746a4a261c9: Verifying Checksum\n2746a4a261c9: Download complete\n69577f69924e: Verifying Checksum\n69577f69924e: Download complete\n9b1ce390f4af: Verifying Checksum\n9b1ce390f4af: Download complete\n35f0abeb72e2: Verifying Checksum\n35f0abeb72e2: Download complete\n21281402473f: Verifying Checksum\n21281402473f: Download complete\nb04aed4ed1be: Verifying Checksum\nb04aed4ed1be: Download complete\nfbc0d5cbe920: Verifying Checksum\nfbc0d5cbe920: Download complete\n4c1d20cdee96: Verifying Checksum\n4c1d20cdee96: Download complete\nba9a469782b4: Verifying Checksum\nba9a469782b4: Download complete\n6e84e72c3ba3: Verifying Checksum\n6e84e72c3ba3: Download complete\n2746a4a261c9: Pull complete\n3baafffdaf65: Verifying Checksum\n3baafffdaf65: Download complete\n4c1d20cdee96: Pull complete\n0d3160e1d0de: Pull complete\nc8e37668deea: Pull complete\n0f17882f5071: Pull complete\nb7f6a1ab3dd4: Verifying Checksum\nb7f6a1ab3dd4: Download complete\nab3f8384cfc6: Pull complete\n9b1ce390f4af: Pull complete\n69577f69924e: Pull complete\n21281402473f: Pull complete\n35f0abeb72e2: Pull complete\nb04aed4ed1be: Pull complete\nba9a469782b4: Pull complete\nfbc0d5cbe920: Pull complete\nb7f6a1ab3dd4: Pull complete\n6e84e72c3ba3: Pull complete\n3baafffdaf65: Pull complete\nDigest: sha256:127f49d1af9832c1b4758e4ee04f6d892c33986c316708faecba12cea0f73669\nStatus: Downloaded newer image for renovate/ruby:2.6.4\n",
rarkins commented 4 years ago

This error is printed if there is no matching end found for a source in a Gemfile. It's impossible to do anything here without more information.

bilby91 commented 4 years ago

@rarkins This is pretty much how the the Gemfile looks. 

source 'https://rubygems.org'

git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '2.7.0'

source 'https://my.private.repo' do
  # Private gems here, they are all ignored for the moment.
end

# Heavy metal SOAP client
gem 'savon', '2.12.0'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 6.0.0'

# Use postgresql as the database for Active Record
gem 'pg', '>= 0.18', '< 2.0'

# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.4.2', require: false

# Use Puma as the app server
gem 'puma', '~> 4.1'

# Code lintern
gem 'rubocop', '~> 0.74.0', require: false

# Exception tracking
gem 'airbrake', '~> 9.4'

# GraphQL Ruby
gem 'graphql', '~> 1.9.10'

# Simple authorization gem for GraphQL
gem 'graphql-guard'

# Paginator for Ruby
gem 'kaminari-activerecord'

# DRY gem for business transactions
gem 'dry-transaction'

# DRY gem for validations
gem 'dry-validation'

# Ruby client for Amazon S3
gem 'aws-sdk-s3', '~> 1'

# Only used to import encryption keys (lib/tasks/encryption_keys.rake)
gem 'rubyzip'

# Simple, efficient background processing for Ruby
gem 'sidekiq', '~> 6.0.2'

# A Kafka producer and consumer
gem 'poseidon', '~> 0.0.5'

gem 'nokogiri'

group :development, :test do
  # Call 'byebug' anywhere in the code to stop execution and get a debugger console
  gem 'byebug', platforms: %i[mri mingw x64_mingw]
  # Security analysis
  gem 'brakeman'
  # Code coverage tool
  gem 'simplecov', require: false
end

group :development do
  gem 'listen', '>= 3.0.5', '< 3.2'
  # Spring speeds up development by keeping your application running in the background.
  # Read more: https://github.com/rails/spring
  gem 'spring'
  gem 'spring-watcher-listen', '~> 2.0.0'
  # Annotate Rails classes with schema and routes info
  gem 'annotate', '~> 2.7.5'
end

group :test do
  gem 'factory_bot', '~> 5.0.2'
  gem 'faker', '~> 2'
  gem 'rspec-rails', '~> 3'
  gem 'shoulda-matchers', '~> 4.1.2'
  gem 'timecop'
  # WebMock allows stubbing HTTP requests and setting expectations on HTTP requests.
  gem 'webmock'
  # RSpec for Sidekiq
  gem 'rspec-sidekiq'
  gem 'simplecov-cobertura'
end

So, based on your previous comment, source 'https://rubygems.org' would be invalid ?

rarkins commented 4 years ago

@bilby91 when I copy/pasted that Gemfile into a test repo, it did not reproduce the error log. If you can reproduce the error in a Gemfile somewhere public, then I can take a look.

bilby91 commented 4 years ago

@rarkins I removed one line that could be causing issues. Will this line be problematic ?

gem 'sidekiq_queue_metrics', git: 'https://github.com/ivanetchart/sidekiq_queue_metrics.git', branch: 'v3.0'
rarkins commented 4 years ago

I have no idea, sorry. Like I said, unless it can be reproduced to test against, there's nothing I can do. It should probably be a warning rather than an error though - Renovate does attempt to continue.

bilby91 commented 4 years ago

@rarkins Sure thing! I'll create a public repo to reproduce the problem and report back! Thanks for the quick response.

bilby91 commented 4 years ago

@rarkins This repo (https://gitlab.com/bilby91/renovate-ruby-source-line-issue) reproduces the error Could not locate Gemfile.

The ERROR: Undefined sourceLine is caused by my private gems piece, that was the only thing I removed from the Gemfile. Since renovate is skipping that error it doesn't seem to be a big deal. Still not sure why it can't locate the Gemfile.

Logs from running renovate:

 Running with gitlab-runner 12.4.1 (05161b14)
   on gitlab-runner-57cdc6fdd7-rfhxc V8CZoqy2
Using Kubernetes namespace: gitlab
00:00
 Using Kubernetes executor with image renovate/renovate:19.109.0 ...
Waiting for pod gitlab/runner-v8czoqy2-project-16504692-concurrent-05xhkp to be running, status is Pending
 Waiting for pod gitlab/runner-v8czoqy2-project-16504692-concurrent-05xhkp to be running, status is Pending
 Waiting for pod gitlab/runner-v8czoqy2-project-16504692-concurrent-05xhkp to be running, status is Pending
 Waiting for pod gitlab/runner-v8czoqy2-project-16504692-concurrent-05xhkp to be running, status is Pending
 Waiting for pod gitlab/runner-v8czoqy2-project-16504692-concurrent-05xhkp to be running, status is Pending
 Running on runner-v8czoqy2-project-16504692-concurrent-05xhkp via gitlab-runner-57cdc6fdd7-rfhxc...
Fetching changes with git depth set to 50...
 Initialized empty Git repository in /builds/renovate/.git/
 Created fresh repository.
 From https://gitlab.com/bilby91/renovate
  * [new ref]         refs/pipelines/112626646 -> refs/pipelines/112626646
  * [new branch]      bundler-bug              -> origin/bundler-bug
 Checking out ba171de5 as bundler-bug...
 Skipping Git submodules setup
$ node /usr/src/app/dist/renovate.js --binary-source=docker
  INFO: Using platform gitAuthor: Martin
  INFO: Renovating repository (repository=bilby91/renovate-ruby-source-line-issue)
        "renovateVersion": "19.109.0"
  INFO: Initialising git repository into /tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue (repository=bilby91/renovate-ruby-source-line-issue)
  INFO: git clone completed (repository=bilby91/renovate-ruby-source-line-issue)
        "cloneSeconds": 1
  INFO: Setting git author (repository=bilby91/renovate-ruby-source-line-issue)
        "gitAuthor": {"name": "Martin", "email": "test@test.com"}
  INFO: Found bundler package files (repository=bilby91/renovate-ruby-source-line-issue)
  INFO: Found dockerfile package files (repository=bilby91/renovate-ruby-source-line-issue)
  INFO: Processing 10 branches: renovate/airbrake-10.x, renovate/annotate-3.x, renovate/docker-ruby-2.x, renovate/factory_bot-5.x, renovate/faker-2.x, renovate/graphql-1.x, renovate/listen-3.x, renovate/rspec-rails-3.x, renovate/rubocop-0.x, renovate/shoulda-matchers-4.x (repository=bilby91/renovate-ruby-source-line-issue)
  INFO: Branch needs creating (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/factory_bot-5.x)
  INFO: Running bundler via docker (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/factory_bot-5.x)
  WARN: Gemfile.lock update failed due to unknown reason - skipping branch (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/factory_bot-5.x)
        "err": {
          "killed": false,
          "code": 10,
          "signal": null,
          "cmd": "docker run --rm -v \"/tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue\":\"/tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue\" -w \"/tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue\" renovate/ruby:2.6.4 bash -l -c \"ruby --version && gem install bundler -v 1.17.2 --no-document && bundle lock --update factory_bot\"",
          "stdout": "ruby 2.6.4p104 (2019-08-28 revision 67798) [x86_64-linux]\nSuccessfully installed bundler-1.17.2\n1 gem installed\nCould not locate Gemfile\n",
          "stderr": "Unable to find image 'renovate/ruby:2.6.4' locally\n2.6.4: Pulling from renovate/ruby\n2746a4a261c9: Pulling fs layer\n4c1d20cdee96: Pulling fs layer\n0d3160e1d0de: Pulling fs layer\nc8e37668deea: Pulling fs layer\n0f17882f5071: Pulling fs layer\nab3f8384cfc6: Pulling fs layer\n9b1ce390f4af: Pulling fs layer\n69577f69924e: Pulling fs layer\n21281402473f: Pulling fs layer\n35f0abeb72e2: Pulling fs layer\nb04aed4ed1be: Pulling fs layer\nba9a469782b4: Pulling fs layer\nfbc0d5cbe920: Pulling fs layer\nb7f6a1ab3dd4: Pulling fs layer\n6e84e72c3ba3: Pulling fs layer\n3baafffdaf65: Pulling fs layer\n21281402473f: Waiting\n35f0abeb72e2: Waiting\nb04aed4ed1be: Waiting\nba9a469782b4: Waiting\nfbc0d5cbe920: Waiting\nb7f6a1ab3dd4: Waiting\n6e84e72c3ba3: Waiting\n3baafffdaf65: Waiting\nc8e37668deea: Waiting\n0f17882f5071: Waiting\nab3f8384cfc6: Waiting\n9b1ce390f4af: Waiting\n69577f69924e: Waiting\n4c1d20cdee96: Verifying Checksum\n4c1d20cdee96: Download complete\n0d3160e1d0de: Verifying Checksum\n0d3160e1d0de: Download complete\nc8e37668deea: Download complete\nab3f8384cfc6: Verifying Checksum\nab3f8384cfc6: Download complete\n0f17882f5071: Verifying Checksum\n0f17882f5071: Download complete\n9b1ce390f4af: Verifying Checksum\n9b1ce390f4af: Download complete\n69577f69924e: Verifying Checksum\n69577f69924e: Download complete\n35f0abeb72e2: Verifying Checksum\n35f0abeb72e2: Download complete\n21281402473f: Verifying Checksum\n21281402473f: Download complete\nb04aed4ed1be: Verifying Checksum\nb04aed4ed1be: Download complete\n2746a4a261c9: Verifying Checksum\n2746a4a261c9: Download complete\nfbc0d5cbe920: Verifying Checksum\nfbc0d5cbe920: Download complete\nb7f6a1ab3dd4: Verifying Checksum\nb7f6a1ab3dd4: Download complete\nba9a469782b4: Verifying Checksum\nba9a469782b4: Download complete\n2746a4a261c9: Pull complete\n4c1d20cdee96: Pull complete\n0d3160e1d0de: Pull complete\nc8e37668deea: Pull complete\n0f17882f5071: Pull complete\nab3f8384cfc6: Pull complete\n9b1ce390f4af: Pull complete\n69577f69924e: Pull complete\n21281402473f: Pull complete\n35f0abeb72e2: Pull complete\nb04aed4ed1be: Pull complete\n6e84e72c3ba3: Verifying Checksum\n6e84e72c3ba3: Download complete\n3baafffdaf65: Verifying Checksum\n3baafffdaf65: Download complete\nba9a469782b4: Pull complete\nfbc0d5cbe920: Pull complete\nb7f6a1ab3dd4: Pull complete\n6e84e72c3ba3: Pull complete\n3baafffdaf65: Pull complete\nDigest: sha256:127f49d1af9832c1b4758e4ee04f6d892c33986c316708faecba12cea0f73669\nStatus: Downloaded newer image for renovate/ruby:2.6.4\n",
          "message": "Command failed: docker run --rm -v \"/tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue\":\"/tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue\" -w \"/tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue\" renovate/ruby:2.6.4 bash -l -c \"ruby --version && gem install bundler -v 1.17.2 --no-document && bundle lock --update factory_bot\"\nUnable to find image 'renovate/ruby:2.6.4' locally\n2.6.4: Pulling from renovate/ruby\n2746a4a261c9: Pulling fs layer\n4c1d20cdee96: Pulling fs layer\n0d3160e1d0de: Pulling fs layer\nc8e37668deea: Pulling fs layer\n0f17882f5071: Pulling fs layer\nab3f8384cfc6: Pulling fs layer\n9b1ce390f4af: Pulling fs layer\n69577f69924e: Pulling fs layer\n21281402473f: Pulling fs layer\n35f0abeb72e2: Pulling fs layer\nb04aed4ed1be: Pulling fs layer\nba9a469782b4: Pulling fs layer\nfbc0d5cbe920: Pulling fs layer\nb7f6a1ab3dd4: Pulling fs layer\n6e84e72c3ba3: Pulling fs layer\n3baafffdaf65: Pulling fs layer\n21281402473f: Waiting\n35f0abeb72e2: Waiting\nb04aed4ed1be: Waiting\nba9a469782b4: Waiting\nfbc0d5cbe920: Waiting\nb7f6a1ab3dd4: Waiting\n6e84e72c3ba3: Waiting\n3baafffdaf65: Waiting\nc8e37668deea: Waiting\n0f17882f5071: Waiting\nab3f8384cfc6: Waiting\n9b1ce390f4af: Waiting\n69577f69924e: Waiting\n4c1d20cdee96: Verifying Checksum\n4c1d20cdee96: Download complete\n0d3160e1d0de: Verifying Checksum\n0d3160e1d0de: Download complete\nc8e37668deea: Download complete\nab3f8384cfc6: Verifying Checksum\nab3f8384cfc6: Download complete\n0f17882f5071: Verifying Checksum\n0f17882f5071: Download complete\n9b1ce390f4af: Verifying Checksum\n9b1ce390f4af: Download complete\n69577f69924e: Verifying Checksum\n69577f69924e: Download complete\n35f0abeb72e2: Verifying Checksum\n35f0abeb72e2: Download complete\n21281402473f: Verifying Checksum\n21281402473f: Download complete\nb04aed4ed1be: Verifying Checksum\nb04aed4ed1be: Download complete\n2746a4a261c9: Verifying Checksum\n2746a4a261c9: Download complete\nfbc0d5cbe920: Verifying Checksum\nfbc0d5cbe920: Download complete\nb7f6a1ab3dd4: Verifying Checksum\nb7f6a1ab3dd4: Download complete\nba9a469782b4: Verifying Checksum\nba9a469782b4: Download complete\n2746a4a261c9: Pull complete\n4c1d20cdee96: Pull complete\n0d3160e1d0de: Pull complete\nc8e37668deea: Pull complete\n0f17882f5071: Pull complete\nab3f8384cfc6: Pull complete\n9b1ce390f4af: Pull complete\n69577f69924e: Pull complete\n21281402473f: Pull complete\n35f0abeb72e2: Pull complete\nb04aed4ed1be: Pull complete\n6e84e72c3ba3: Verifying Checksum\n6e84e72c3ba3: Download complete\n3baafffdaf65: Verifying Checksum\n3baafffdaf65: Download complete\nba9a469782b4: Pull complete\nfbc0d5cbe920: Pull complete\nb7f6a1ab3dd4: Pull complete\n6e84e72c3ba3: Pull complete\n3baafffdaf65: Pull complete\nDigest: sha256:127f49d1af9832c1b4758e4ee04f6d892c33986c316708faecba12cea0f73669\nStatus: Downloaded newer image for renovate/ruby:2.6.4\n",
          "stack": "Error: Command failed: docker run --rm -v \"/tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue\":\"/tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue\" -w \"/tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue\" renovate/ruby:2.6.4 bash -l -c \"ruby --version && gem install bundler -v 1.17.2 --no-document && bundle lock --update factory_bot\"\nUnable to find image 'renovate/ruby:2.6.4' locally\n2.6.4: Pulling from renovate/ruby\n2746a4a261c9: Pulling fs layer\n4c1d20cdee96: Pulling fs layer\n0d3160e1d0de: Pulling fs layer\nc8e37668deea: Pulling fs layer\n0f17882f5071: Pulling fs layer\nab3f8384cfc6: Pulling fs layer\n9b1ce390f4af: Pulling fs layer\n69577f69924e: Pulling fs layer\n21281402473f: Pulling fs layer\n35f0abeb72e2: Pulling fs layer\nb04aed4ed1be: Pulling fs layer\nba9a469782b4: Pulling fs layer\nfbc0d5cbe920: Pulling fs layer\nb7f6a1ab3dd4: Pulling fs layer\n6e84e72c3ba3: Pulling fs layer\n3baafffdaf65: Pulling fs layer\n21281402473f: Waiting\n35f0abeb72e2: Waiting\nb04aed4ed1be: Waiting\nba9a469782b4: Waiting\nfbc0d5cbe920: Waiting\nb7f6a1ab3dd4: Waiting\n6e84e72c3ba3: Waiting\n3baafffdaf65: Waiting\nc8e37668deea: Waiting\n0f17882f5071: Waiting\nab3f8384cfc6: Waiting\n9b1ce390f4af: Waiting\n69577f69924e: Waiting\n4c1d20cdee96: Verifying Checksum\n4c1d20cdee96: Download complete\n0d3160e1d0de: Verifying Checksum\n0d3160e1d0de: Download complete\nc8e37668deea: Download complete\nab3f8384cfc6: Verifying Checksum\nab3f8384cfc6: Download complete\n0f17882f5071: Verifying Checksum\n0f17882f5071: Download complete\n9b1ce390f4af: Verifying Checksum\n9b1ce390f4af: Download complete\n69577f69924e: Verifying Checksum\n69577f69924e: Download complete\n35f0abeb72e2: Verifying Checksum\n35f0abeb72e2: Download complete\n21281402473f: Verifying Checksum\n21281402473f: Download complete\nb04aed4ed1be: Verifying Checksum\nb04aed4ed1be: Download complete\n2746a4a261c9: Verifying Checksum\n2746a4a261c9: Download complete\nfbc0d5cbe920: Verifying Checksum\nfbc0d5cbe920: Download complete\nb7f6a1ab3dd4: Verifying Checksum\nb7f6a1ab3dd4: Download complete\nba9a469782b4: Verifying Checksum\nba9a469782b4: Download complete\n2746a4a261c9: Pull complete\n4c1d20cdee96: Pull complete\n0d3160e1d0de: Pull complete\nc8e37668deea: Pull complete\n0f17882f5071: Pull complete\nab3f8384cfc6: Pull complete\n9b1ce390f4af: Pull complete\n69577f69924e: Pull complete\n21281402473f: Pull complete\n35f0abeb72e2: Pull complete\nb04aed4ed1be: Pull complete\n6e84e72c3ba3: Verifying Checksum\n6e84e72c3ba3: Download complete\n3baafffdaf65: Verifying Checksum\n3baafffdaf65: Download complete\nba9a469782b4: Pull complete\nfbc0d5cbe920: Pull complete\nb7f6a1ab3dd4: Pull complete\n6e84e72c3ba3: Pull complete\n3baafffdaf65: Pull complete\nDigest: sha256:127f49d1af9832c1b4758e4ee04f6d892c33986c316708faecba12cea0f73669\nStatus: Downloaded newer image for renovate/ruby:2.6.4\n\n    at ChildProcess.exithandler (child_process.js:294:12)\n    at ChildProcess.emit (events.js:198:13)\n    at ChildProcess.EventEmitter.emit (domain.js:448:20)\n    at maybeClose (internal/child_process.js:982:16)\n    at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)"
        }
  INFO: Branch needs creating (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/faker-2.x)
  INFO: Aborting Bundler artifacts due to previous failed attempt (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/faker-2.x)
  INFO: Branch needs creating (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/graphql-1.x)
  INFO: Aborting Bundler artifacts due to previous failed attempt (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/graphql-1.x)
  INFO: Branch needs creating (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/listen-3.x)
  INFO: Aborting Bundler artifacts due to previous failed attempt (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/listen-3.x)
  INFO: Branch needs creating (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/rspec-rails-3.x)
  INFO: Aborting Bundler artifacts due to previous failed attempt (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/rspec-rails-3.x)
  INFO: Branch needs creating (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/rubocop-0.x)
  INFO: Aborting Bundler artifacts due to previous failed attempt (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/rubocop-0.x)
  INFO: Branch needs creating (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/shoulda-matchers-4.x)
  INFO: Aborting Bundler artifacts due to previous failed attempt (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/shoulda-matchers-4.x)
  INFO: Branch needs creating (repository=bilby91/renovate-ruby-source-line-issue, branch=renovate/docker-ruby-2.x)
  INFO: Repository has changed during renovation - aborting (repository=bilby91/renovate-ruby-source-line-issue)
  INFO: Finished repository (repository=bilby91/renovate-ruby-source-line-issue)
  INFO: Renovate finished
 Job succeeded
rarkins commented 4 years ago

I've copy and pasted that Gemfile into https://github.com/renovate-tests/bundler8/blob/master/Gemfile and it does not reproduce the problem when run against Renovate master

bilby91 commented 4 years ago

How is the test running ? I'm using binarySource=docker BTW. Wouldn't the lockfile be required as well for the test ?

rarkins commented 4 years ago

This error is logged from the extract phase, which is prior to any binaries being called. I also run binarySource=docker locally.

rarkins commented 4 years ago 
$ yarn start bilby91/renovate-ruby-source-line-issue --dry-run=true --log-level=info
yarn run v1.21.1
$ babel-node --extensions ".ts,.js" -- lib/renovate.ts bilby91/renovate-ruby-source-line-issue --dry-run=true --log-level=info
 INFO: Using default GitLab endpoint: https://gitlab.com/api/v4/
 INFO: Using platform gitAuthor: Renovate Tests <renovate-tests@redgc.com>
 INFO: Renovating repository (repository=bilby91/renovate-ruby-source-line-issue)
       "renovateVersion": "0.0.0-semantic-release"
 INFO: Initialising git repository into /tmp/renovate/repos/gitlab/bilby91/renovate-ruby-source-line-issue (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: git clone completed (repository=bilby91/renovate-ruby-source-line-issue)
       "cloneSeconds": 1.3
 INFO: Setting git author (repository=bilby91/renovate-ruby-source-line-issue)
       "gitAuthor": {"name": "Renovate Tests", "email": "renovate-tests@redgc.com"}
 INFO: Repo is not onboarded (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Found bundler package files (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Found dockerfile package files (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Found gitlabci package files (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Need to create onboarding PR (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Creating onboarding branch (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: DRY-RUN: Would commit files to onboarding branch (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Found bundler package files (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Found dockerfile package files (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Found gitlabci package files (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Creating onboarding PR (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: DRY-RUN: Would create onboarding PR (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Finished repository (repository=bilby91/renovate-ruby-source-line-issue)
 INFO: Renovate finished
Done in 13.36s.
bilby91 commented 4 years ago

@rarkins I didn't have issues when running --dry-run nor before the onboarding.

Can you try without --dry-run and using onboarding: false and requireConfig: false ?

rarkins commented 4 years ago

@bilby91 I ran it completely through without dry run previously and it created PRs without error: https://github.com/renovate-tests/bundler8/pulls

The only reason for running with dry run was because I ran it on your actual gitlab repository, which I obviously don't have access rights to.

bilby91 commented 4 years ago

Might be a silly question but, could it be possible that the missing Gemfile.lock on the renovate-tests/bundler8 repo could be triggering a different behaviour ?

rarkins commented 4 years ago

I added the Gemfile.lock to check that it's not the cause: https://github.com/renovate-tests/bundler8/pull/3/files

bilby91 commented 4 years ago

@rarkins The problem seems to be that the repository directory is empty after cloning. At least I can't find any files after running renovate under /tmp/renovate/repos/gitlab/my/repo/path

bilby91 commented 4 years ago

@rarkins I think I finally traced the root problem. I have some missing volume mounting with the docker in docker environment. I'll report back once I can get this working and potentially close the issue!

bilby91 commented 4 years ago

@rarkins I think it's now working correctly except that I'm now hitting this issue: https://github.com/renovatebot/renovate/issues/5023

The private gems sections requires the bundle config call. I'm already passing the hostRules correctly.

rarkins commented 4 years ago

@bilby91 that's great. Does that mean we can close this issue?

bilby91 commented 4 years ago

Yes, let's close this one!

bilby91 commented 4 years ago

@rarkins I'm re opening this one since I think I found the issue. This doesn't seem to be supported:

gem 'my-gem', '0.2.1', require: 'my_gem'
rarkins commented 4 years ago

Can you reproduce in a public repo Incan test against?

bilby91 commented 4 years ago

@rarkins The problem is interesting, I though it was related to the require but it isn't.

This works fine:

# frozen_string_literal: true

source 'https://rubygems.org' do
  gem 'rubocop'
  gem 'brakeman'
end

This doesn't

# frozen_string_literal: true

source 'https://rubygems.org' do
  gem 'rubocop'

  gem 'brakeman'
end

New lines on the middle of a source block seems to cause problems.

bilby91 commented 4 years ago

@rarkins This PR should fix the issue => https://github.com/renovatebot/renovate/pull/5270

renovate-release commented 4 years ago

:tada: This issue has been resolved in version 19.116.5 :tada:

The release is available on:

Your semantic-release bot :package::rocket: