search

dependabot / cli

A tool for testing and debugging Dependabot update jobs.
MIT License
220 stars 35 forks source link

failed to update deps from auth failures results in failed build in javascript #246

Closed sblatnick closed 7 months ago

sblatnick commented 8 months ago

Command used:

/opt/dependabot update -f job.yaml --local ./ -o dependabot-results.yaml

Config used:

job:
    package-manager: npm_and_yarn
    allowed-updates:
      - update-type: all
    source:
        provider: github
        repo: local/scan
        directory: /
credentials:
  - type: npm_registry
    registry: https://nexus.redacted.org/npm-all/
    username: redacted
    password: redacted

Logs:

08:54:39  dependabot:   CLI: /opt/dependabot update -f job.yaml --local ./ -o ${_RESULTS}
08:54:39  [Pipeline] sh
08:54:39  + /opt/dependabot update -f job.yaml --local ./ -o dependabot-results.yaml
08:54:39      cli | 2024/01/22 15:47:04 Inserting $LOCAL_GITHUB_ACCESS_TOKEN into credentials
08:54:39      cli | 2024/01/22 15:47:04 pulling image: ghcr.io/github/dependabot-update-job-proxy/dependabot-update-job-proxy:latest
08:54:41      cli | 2024/01/22 15:47:06 using image ghcr.io/github/dependabot-update-job-proxy/dependabot-update-job-proxy:latest at sha256:0407f9d3061fe12170111e36b0298d0beac847c5accdd221f17d3d1c28364ddf
08:54:41      cli | 2024/01/22 15:47:06 pulling image: ghcr.io/dependabot/dependabot-updater-npm
08:54:53      cli | 2024/01/22 15:47:18 using image ghcr.io/dependabot/dependabot-updater-npm at sha256:c8c84c8e7323311347af43e92cdfb990bbbcf17560bf5b87f4a955751dba9f50
08:55:05    proxy | 2024/01/22 15:47:29 proxy starting, commit: ce669fe3098a0bddfad98850916eaecfa799dfde
08:55:05    proxy | 2024/01/22 15:47:29 initializing metrics client: No address passed and autodetection from environment failed
08:55:05    proxy | 2024/01/22 15:47:29 Listening (:1080)
08:55:15  updater | Reinitialized existing Git repository in /home/dependabot/dependabot-updater/repo/.git/
08:55:27  updater | 🔍  Finding changed files since git revision c8abcc439.
08:55:35  updater | 🎯  Found 1973 changed files.
08:55:35  updater | ✍️  Fixing up codeql/.codeqlmanifest.json.
08:55:35  updater | ✍️  Fixing up codeql/LICENSE.md.
...
08:56:10  updater | ✍️  Fixing up codeql/qlpacks/codeql/javascript-queries/0.8.6/.codeql/libraries/codeql/dataflow/0.1.6/CHANGELOG.md.
...
08:56:15  updater | /home/dependabot/dependabot-updater/repo/node_modules/prettier/index.js:7348
08:56:15  updater |           throw error;
08:56:15  updater |           ^
08:56:15  updater | 
08:56:15  updater | SyntaxError: All collection items must start at the same column (5:5)
08:56:15  updater |    3 | jobs:
08:56:15  updater |    4 |   echo-body:
08:56:15  updater | >  5 |     runs-on: ubuntu-latest
08:56:15  updater |      |     ^^^^^^^^^^^^^^^^^^^^^^
08:56:15  updater | >  6 |     steps:
08:56:15  updater |      | ^^^^^^^^^^
08:56:15  updater | >  7 |     -  env:
08:56:15  updater |      | ^^^^^^^^^^
08:56:15  updater | >  8 |         BODY: ${{ github.event.issue.body }}
08:56:15  updater |      | ^^^^^^^^^^
08:56:15  updater | >  9 |       run: |
08:56:15  updater |      | ^^^^^^^^^^
08:56:15  updater | > 10 |         echo '${{ env.BODY }}'
08:56:15  updater |      | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
08:56:15  updater |     at n (/home/dependabot/dependabot-updater/repo/node_modules/prettier/parser-yaml.js:1:1125)
08:56:15  updater |     at Object.lr [as parse] (/home/dependabot/dependabot-updater/repo/node_modules/prettier/parser-yaml.js:150:3774)
08:56:15  updater |     at Object.parse (/home/dependabot/dependabot-updater/repo/node_modules/prettier/index.js:7334:23)
08:56:15  updater |     at coreFormat (/home/dependabot/dependabot-updater/repo/node_modules/prettier/index.js:8645:18)
08:56:15  updater |     at formatWithCursor2 (/home/dependabot/dependabot-updater/repo/node_modules/prettier/index.js:8837:18)
08:56:15  updater |     at /home/dependabot/dependabot-updater/repo/node_modules/prettier/index.js:37229:12
08:56:15  updater |     at Object.format (/home/dependabot/dependabot-updater/repo/node_modules/prettier/index.js:37243:12)
08:56:15  updater |     at _default (/home/dependabot/dependabot-updater/repo/node_modules/pretty-quick/dist/processFiles.js:42:29)
08:56:15  updater |     at _default (/home/dependabot/dependabot-updater/repo/node_modules/pretty-quick/dist/index.js:60:29)
08:56:15  updater |     at Object.<anonymous> (/home/dependabot/dependabot-updater/repo/node_modules/pretty-quick/bin/pretty-quick.js:17:27) {
08:56:15  updater |   loc: {
08:56:15  updater |     start: { offset: 42, line: 5, column: 5 },
08:56:15  updater |     end: { offset: 176, line: 10, column: 31 }
08:56:15  updater |   },
08:56:15  updater |   codeFrame: '   3 | jobs:\n' +
08:56:15  updater |     '   4 |   echo-body:\n' +
08:56:15  updater |     '>  5 |     runs-on: ubuntu-latest\n' +
08:56:15  updater |     '     |     ^^^^^^^^^^^^^^^^^^^^^^\n' +
08:56:15  updater |     '>  6 |     steps:\n' +
08:56:15  updater |     '     | ^^^^^^^^^^\n' +
08:56:15  updater |     '>  7 |     -  env:\n' +
08:56:15  updater |     '     | ^^^^^^^^^^\n' +
08:56:15  updater |     '>  8 |         BODY: ${{ github.event.issue.body }}\n' +
08:56:15  updater |     '     | ^^^^^^^^^^\n' +
08:56:15  updater |     '>  9 |       run: |\n' +
08:56:15  updater |     '     | ^^^^^^^^^^\n' +
08:56:15  updater |     "> 10 |         echo '${{ env.BODY }}'\n" +
08:56:15  updater |     '     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^'
08:56:15  updater | }
08:56:15  updater | 
08:56:15  updater | Node.js v18.19.0
08:56:15  updater | husky - pre-commit hook exited with code 1 (error)
08:56:15  updater | Updating certificates in /etc/ssl/certs...
08:56:16  updater | rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
08:56:16  updater | 1 added, 0 removed; done.
08:56:16  updater | Running hooks in /etc/ca-certificates/update.d...
08:56:16  updater | done.
08:56:16  updater | 2024/01/22 15:48:41 INFO Raven 3.1.2 configured not to capture errors: DSN not set
08:56:17  updater | 2024/01/22 15:48:42 INFO Starting job processing
08:56:17  updater | 2024/01/22 15:48:42 INFO Finished job processing
08:56:18  updater | 2024/01/22 15:48:43 INFO Raven 3.1.2 configured not to capture errors: DSN not set
08:56:18  updater | 2024/01/22 15:48:43 INFO Starting job processing
08:56:19    proxy | 2024/01/22 15:48:44 [001] POST http://host.docker.internal:8088/update_jobs/cli/update_dependency_list
08:56:19  {"data":{"dependencies":[{"name":"@angular/animations","requirements":[{"file":"package.json","groups":["dependencies"],"requirement":"^16.2.11","source":{"type":"registry","url":"https://nexus.redacted.com/repository/npm-all"}}],"version":"16.2.11"},{"name":"@angular/common","requirements":[{"file":"package.json","groups":["dependencies"],"requirement":"^16.2.11","source":{"type":"registry","url":"https://nexus.redacted.com/repository/npm-all"}}],"version":"16.2.11"}...
08:56:19    proxy | 2024/01/22 15:48:44 [001] 200 http://host.docker.internal:8088/update_jobs/cli/update_dependency_list
08:56:19    proxy | 2024/01/22 15:48:44 [002] POST http://host.docker.internal:8088/update_jobs/cli/increment_metric
08:56:19  {"data":{"metric":"updater.started","tags":{"operation":"update_all_versions"}},"type":"increment_metric"}
08:56:19    proxy | 2024/01/22 15:48:44 [002] 200 http://host.docker.internal:8088/update_jobs/cli/increment_metric
08:56:19  updater | 2024/01/22 15:48:44 INFO Starting update job for local/scan
08:56:19  updater | 2024/01/22 15:48:44 INFO Checking all dependencies for version updates...
08:56:19  updater | 2024/01/22 15:48:44 INFO Checking if @angular/animations 16.2.11 needs updating
08:56:19    proxy | 2024/01/22 15:48:44 [004] GET https://nexus.redacted.com:443/repository/npm-all/@angular%2Fanimations
08:56:19    proxy | 2024/01/22 15:48:44 [004] * authenticating npm registry request (host: nexus.redacted.com, token auth)
08:56:20    proxy | 2024/01/22 15:48:45 [004] 401 https://nexus.redacted.com:443/repository/npm-all/@angular%2Fanimations
08:56:20    proxy | 2024/01/22 15:48:45 [005] POST http://host.docker.internal:8088/update_jobs/cli/record_update_job_error
08:56:20  {"data":{"error-type":"private_source_authentication_failure","error-details":{"source":"https://nexus.redacted.com/repository/npm-all/"}},"type":"record_update_job_error"}
08:56:20    proxy | 2024/01/22 15:48:45 [005] 200 http://host.docker.internal:8088/update_jobs/cli/record_update_job_error
08:56:20  updater | 2024/01/22 15:48:45 INFO Handled error whilst updating @angular/animations: private_source_authentication_failure {:source=>"https://nexus.redacted.com/repository/npm-all/"}
08:56:20  updater | 2024/01/22 15:48:45 INFO Checking if @angular/common 16.2.11 needs updating
08:56:20    proxy | 2024/01/22 15:48:45 [007] GET https://nexus.redacted.com:443/repository/npm-all/@angular%2Fcommon
08:56:20    proxy | 2024/01/22 15:48:45 [007] * authenticating npm registry request (host: nexus.redacted.com, token auth)
08:56:21    proxy | 2024/01/22 15:48:46 [007] 401 https://nexus.redacted.com:443/repository/npm-all/@angular%2Fcommon
08:56:21    proxy | 2024/01/22 15:48:46 [008] POST http://host.docker.internal:8088/update_jobs/cli/record_update_job_error
08:56:21  {"data":{"error-type":"private_source_authentication_failure","error-details":{"source":"https://nexus.redacted.com/repository/npm-all/"}},"type":"record_update_job_error"}
08:56:21    proxy | 2024/01/22 15:48:46 [008] 200 http://host.docker.internal:8088/update_jobs/cli/record_update_job_error
08:56:21  updater | 2024/01/22 15:48:46 INFO Handled error whilst updating @angular/common: private_source_authentication_failure {:source=>"https://nexus.redacted.com/repository/npm-all/"}
08:56:21  updater | 2024/01/22 15:48:46 INFO Checking if @angular/compiler 16.2.11 needs updating
08:56:21    proxy | 2024/01/22 15:48:46 [010] GET https://nexus.redacted.com:443/repository/npm-all/@angular%2Fcompiler
08:56:21    proxy | 2024/01/22 15:48:46 [010] * authenticating npm registry request (host: nexus.redacted.com, token auth)
08:56:21    proxy | 2024/01/22 15:48:46 [010] 401 https://nexus.redacted.com:443/repository/npm-all/@angular%2Fcompiler
08:56:21    proxy | 2024/01/22 15:48:47 [011] POST http://host.docker.internal:8088/update_jobs/cli/record_update_job_error
08:56:21  {"data":{"error-type":"private_source_authentication_failure","error-details":{"source":"https://nexus.redacted.com/repository/npm-all/"}},"type":"record_update_job_error"}
08:56:21    proxy | 2024/01/22 15:48:47 [011] 200 http://host.docker.internal:8088/update_jobs/cli/record_update_job_error
...
09:03:10  updater | 2024/01/22 15:55:35 INFO Finished job processing
09:03:10  updater | 2024/01/22 15:55:35 INFO Results:
09:03:10  updater | Dependabot encountered '1119' error(s) during execution, please check the logs for more details.
09:03:10  updater | +-----------------------------------------------------------------------------------------------------------------------+
09:03:10  updater | |                                             Dependencies failed to update                                             |
09:03:10  updater | +-------------------------------------------------------------------------------+---------------------------------------+
09:03:10  updater | | @angular/animations                                                           | private_source_authentication_failure |
09:03:10  updater | | @angular/common                                                               | private_source_authentication_failure |
09:03:10  updater | | @angular/compiler                                                             | private_source_authentication_failure |
09:03:10  updater | | @angular/core                                                                 | private_source_authentication_failure |
09:03:10  updater | | @angular/forms                                                                | private_source_authentication_failure |
...
09:03:10  updater | | y18n                                                                          | private_source_authentication_failure |
09:03:10  updater | | yauzl                                                                         | private_source_authentication_failure |
09:03:10  updater | | yn                                                                            | private_source_authentication_failure |
09:03:10  updater | +-------------------------------------------------------------------------------+---------------------------------------+
09:03:12    proxy | 2024/01/22 15:55:37 0/1119 calls cached (0%)
09:03:13      cli | 2024/01/22 15:55:38 updater failure: updater exited with code 1
09:03:13  [Pipeline] }
09:03:13  [Pipeline] // withDockerRegistry
09:03:13  [Pipeline] }
09:03:13  [Pipeline] // withEnv
09:03:13  [Pipeline] }
09:03:13  [Pipeline] // script
09:03:13  [Pipeline] }
09:03:13  [Pipeline] // container
09:03:13  [Pipeline] echo
...
09:03:14  [Pipeline] // podTemplate
09:03:14  [Pipeline] End of Pipeline
09:03:14  ERROR: Stopping pipeline
09:03:14  [Bitbucket] Notifying pull request build result
09:03:15  [Bitbucket] Build result notified
09:03:15  Finished: FAILURE

I did notice from the logs the url passed in yaml is different than the one in the logs, by having "repository" at the top level.

See also #230 as potentially related.

jakecoffman commented 8 months ago

Try using token defined as $username:$password:

credentials:
  - type: npm_registry
    registry: https://nexus.redacted.org/npm-all/
    token: $token

It's a long story why it's defined this way, but we should do better validating the correct properties are present rather than accepting anything.

sblatnick commented 8 months ago

Should it be base64 encoded or not?

jakecoffman commented 8 months ago

It should not be.

sblatnick commented 8 months ago

Good news! That appears to have fixed the error. Thanks!