search

github / gh-valet

Valet helps facilitate the migration of Azure DevOps, CircleCI, GitLab CI, Jenkins, and Travis CI pipelines to GitHub Actions.
MIT License
510 stars 35 forks source link

[Support]: undefined method `[]' for nil:NilClass #63

Closed digimangos closed 2 years ago

digimangos commented 2 years ago

What operating system are you using?

What version of the tool are you using?

0.1.0.13168 (27eaf8ab1f68c1c9dcfcf665a52ba6f1a5686504)

What happened?

When executing an audit of pipelines from ADO, several pipelines fail with the following error: Message: undefined method `[]' for nil:NilClass

I believe the customer is executing this in windows, but using WSL prompt.

Relevant log output

There was an error extracting the Azure DevOps pipeline.
Message: undefined method `[]' for nil:NilClass
/usr/local/bundle/gems/valet-0.1.0.13168/lib/valet/services/azure_devops/pipeline/extract_pipeline.rb:46:in `extract'
/usr/local/bundle/gems/valet-0.1.0.13168/lib/valet/services/azure_devops/extract_all_pipelines.rb:20:in `block (3 levels) in call'
/usr/local/bundle/gems/valet-0.1.0.13168/lib/valet/models/thread_collection.rb:7:in `block in perform_later'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:24:in `block in execute'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:47:in `block in synchronize'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:47:in `synchronize'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:47:in `synchronize'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:22:in `execute'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/ivar.rb:169:in `safe_execute'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/future.rb:55:in `block in execute'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:352:in `run_task'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:343:in `block (3 levels) in create_worker'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `loop'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in `block (2 levels) in create_worker'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in `catch'
/usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in `block in create_worker'

Code of Conduct

begonaguereca commented 2 years ago

Hey @digimangos! Ok so after taking a look at the error it looks like we are not gracefully handling a case where Valet is not finding the pipeline. Could you confirm with the client that the following pipelines exist? 

Valet Audit Error Catcher

Returning pipelines containing error: 'undefined method `[]' for nil:NilClass'

Total pipelines with this error: 8
Run on: 2022-07-14 07:43:08 -0700

1. Error message: There was an error extracting the Azure DevOps pipeline.
Message: undefined method `[]' for nil:NilClass
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-TestCafe-E2E.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=293

2. Error message: There was an error extracting the Azure DevOps pipeline.
Message: undefined method `[]' for nil:NilClass
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-Backend-BusinessIntelligenceService.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=332

3. Error message: There was an error extracting the Azure DevOps pipeline.
Message: undefined method `[]' for nil:NilClass
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-CRM.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=318

4. Error message: There was an error extracting the Azure DevOps pipeline.
Message: undefined method `[]' for nil:NilClass
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-BackOffice-Frontend.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=374

5. Error message: There was an error extracting the Azure DevOps pipeline.
Message: undefined method `[]' for nil:NilClass
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-Developers-Static.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=376

6. Error message: There was an error extracting the Azure DevOps pipeline.
Message: undefined method `[]' for nil:NilClass
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.PushNotifications.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=335

7. Error message: There was an error extracting the Azure DevOps pipeline.
Message: undefined method `[]' for nil:NilClass
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Backend-MailImporter.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=353

8. Error message: There was an error extracting the Azure DevOps pipeline.
Message: undefined method `[]' for nil:NilClass
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-Docs.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=366
begonaguereca commented 2 years ago

I also noticed that there were 24 instances where Valet couldn't find GitHub credentials to pull pipelines who's source code is in GitHub. Could you also confirm that a GitHub token was provided?

If there are different tokens that are needed to access pipelines in different orgs within GitHub, be sure to have them follow these instructions to create a credentials file. These are the pipelines that had credentials errors: 

Valet Audit Error Catcher

Returning pipelines containing error: locate credentials

Total pipelines with this error: 24
Run on: 2022-07-14 08:06:10 -0700

1. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/lambda-dotnet-templates)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Lambda-DotNetTemplates.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=365

2. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/***-Cloudformation-Templates.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Cloudformation-Templates-Cognito.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=255

3. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-Integration-Events)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-Integration-Events.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=330

4. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-Monitoring-Plugins)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Monitoring-Plugins.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=297

5. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-PublicApiGateway)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-PublicApiGateway.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=257

6. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/***-Basebox.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Basebox.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=135

7. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/***-Backend-Components-OAuth2.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-Backend-Components-OAuth2.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=243

8. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-Backend-Onboarding)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Backend-Onboarding.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=373

9. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/***-IaC-EventProcessing.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-iac-eventprocessing.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=234

10. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-Backend-SearchService)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Backend-SearchService.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=276

11. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-FeatureToggleService)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-FeatureToggleService.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=347

12. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/dotnetcore-sample.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Testing-YAML file.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=103

13. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/***-Cloudformation-Templates.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Cloudformation-Templates-VpcEndpoints.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=259

14. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-Logging)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-Logging.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=300

15. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-PostingRules-API)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-PostingRulesAPI.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=343

16. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/***-Backend-DocumentArtifact.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-Backend-DocumentArtifact.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=298

17. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-Backend-Components-UblProcessor-Contracts)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Backend-Components-UblProcessor-Contracts.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=349

18. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/***-Backend-Orchestration.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-Backend-Orchestration.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=247

19. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/***-LegalDocuments-Service.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.***-LegalDocuments-Service.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=253

20. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***.UblProcessor.Service)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***.UblProcessor.Service.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=346

21. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-Backend-Company)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Backend-Company.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=284

22. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-Backend-UserService)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Backend-UserService.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=338

23. Error message: There was an error extracting the Azure DevOps pipeline.
Message: Unable to locate credentials for 'https://github.com/***/***-Backend-Components-S3Client.git'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Backend-Components-S3Client.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=246

24. Error message: There was an error extracting the Azure DevOps pipeline.
Message: There was an error retrieving file contents from GitHub (***/***-Backend-Robotic-Accounting)
Message: Unable to locate credentials for 'https://github.com'
File name: /Users/bguereca/Downloads/***-main/valet-pipeline-analysis/results-20220629/***/***-Robotic-Accounting.error.txt
URL: https://dev.azure.com/***/***/_build/definition?definitionId=326
digimangos commented 2 years ago

@begonaguereca, I've asked for feedback regarding the first issue.

Yes we created a credential file. I thought most of these errors had disappeared because they were a lot worse before creating the file. Hmmm, I can look in to that with them, perhaps the user running the audit doesn't have full access.

begonaguereca commented 2 years ago

@digimangos closing the issue for now since your engagement with this client is over!