dependabot / smoke-tests

A collection of manifest files for various package managers and is used to perform end-to-end tests for Dependabot.
39 stars 22 forks source link

Update maven smoke test to produce correct output #226

Closed amazimbe closed 1 month ago

amazimbe commented 1 month ago

PR link: https://github.com/dependabot/smoke-tests/pull/225

This test: https://github.com/dependabot/smoke-tests/blob/main/tests/smoke-maven-group-multidir.yaml currently passes because it wrongly considers version 1.5.11.RELEASE to be lower than 1.5.11. However, I've fixed the parsing on my PR: https://github.com/dependabot/dependabot-core/pull/10558 and as a result this test is now failing.

This ticket is for the work to update the test.

amazimbe commented 1 month ago

I've checked out the smoke tests repo: https://github.com/dependabot/smoke-tests but when I run

script/dependabot test -f tests/smoke-maven-group-multidir.yaml -o output.yaml

on the main branch as well as on my feature branch and I'm getting an error:

proxy | 2024/09/11 09:10:16 [063] POST http://host.docker.internal:52059/update_jobs/cli/create_pull_request
  cli | 2024/09/11 09:10:16 yaml: unmarshal errors: line 1: field directory not found in type model.Dependency line 1: field directory not found in type model.Dependency
  cli | 2024/09/11 09:10:16 http: panic serving 127.0.0.1:52106: yaml: unmarshal errors:
 line 1: field directory not found in type model.Dependency
 line 1: field directory not found in type model.Dependency
goroutine 29 [running]:
net/http.(*conn).serve.func1()
    /usr/local/go/src/net/http/server.go:1898 +0xbe
panic({0x49dc9e0?, 0xc000376dc8?})
    /usr/local/go/src/runtime/panic.go:770 +0x132
github.com/dependabot/cli/internal/server.(*API).assertExpectation(0xc0001b8b48, {0xc0000bc3d1, 0x13}, 0xc0000307d0)
    /github/workspace/internal/server/api.go:170 +0x1ec
github.com/dependabot/cli/internal/server.(*API).ServeHTTP(0xc0001b8b48, {0x4a80b20, 0xc00053a1c0}, 0xc0003b6900)
    /github/workspace/internal/server/api.go:147 +0x37b
net/http.serverHandler.ServeHTTP({0x4a7f560?}, {0x4a80b20?, 0xc00053a1c0?}, 0x6?)
    /usr/local/go/src/net/http/server.go:3137 +0x8e
net/http.(*conn).serve(0xc00038f710, {0x4a81d68, 0xc0000a00f0})
    /usr/local/go/src/net/http/server.go:2039 +0x5e8
created by net/http.(*Server).Serve in goroutine 20
    /usr/local/go/src/net/http/server.go:3285 +0x4b4
 proxy | 2024/09/11 09:10:16 [063] No response from server
 proxy | 2024/09/11 09:10:16 [063] No response from server
 proxy | 2024/09/11 09:10:16 [064] POST http://host.docker.internal:52059/update_jobs/cli/record_update_job_error
  cli | 2024/09/11 09:10:16 type was unexpected: expected mark_as_processed got record_update_job_error
 proxy | 2024/09/11 09:10:16 [064] 200 http://host.docker.internal:52059/update_jobs/cli/record_update_job_error
updater | 2024/09/11 09:10:16 ERROR Error processing job (Dependabot::ApiError)
updater | 2024/09/11 09:10:16 ERROR EOF
updater | 
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/api_client.rb:42:in `block in create_pull_request'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace/tracer.rb:37:in `block in in_span'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace.rb:70:in `block in with_span'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/context.rb:87:in `with_value'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace.rb:70:in `with_span'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace/tracer.rb:37:in `in_span'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/api_client.rb:34:in `create_pull_request'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `bind_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `validate_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/service.rb:57:in `create_pull_request'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `bind_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `validate_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/create_group_update_pull_request.rb:64:in `perform'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `bind_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `validate_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/group_update_all_versions.rb:117:in `run_grouped_update_for'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `bind_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `validate_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/group_update_all_versions.rb:103:in `block in run_grouped_dependency_updates'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/group_update_all_versions.rb:102:in `each'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/group_update_all_versions.rb:102:in `run_grouped_dependency_updates'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `bind_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `validate_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/group_update_all_versions.rb:64:in `perform'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `bind_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/call_validation.rb:270:in `validate_call'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11444/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:45:in `run'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:44:in `block in perform_job'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace/tracer.rb:37:in `block in in_span'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace.rb:70:in `block in with_span'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/context.rb:87:in `with_value'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace.rb:70:in `with_span'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/opentelemetry-api-1.2.3/lib/opentelemetry/trace/tracer.rb:37:in `in_span'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:18:in `perform_job'
updater | 2024/09/11 09:10:16 ERROR /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:37:in `run'
updater | 2024/09/11 09:10:16 ERROR bin/update_files.rb:47:in `<main>'
updater | 2024/09/11 09:10:16 INFO Found no dependencies to update after filtering allowed updates in /maven/multi-dir/foo
amazimbe commented 1 month ago

The generated output file is missing a lo of data

input:
    job:
        package-manager: maven
        allowed-updates:
            - dependency-type: direct
              update-type: all
        dependency-groups:
            - name: maven
              applies-to: security-updates
              rules:
                patterns:
                    - '*'
        dependencies:
            - org.springframework.boot:spring-boot-starter-parent
            - org.springframework.boot:spring-boot-starter-parent
        ignore-conditions:
            - dependency-name: org.springframework.boot:spring-boot-starter-parent
              source: tests/smoke-maven-group-multidir.yaml
              version-requirement: '>1.5.12.RELEASE'
            - dependency-name: org.springframework.boot:spring-boot-starter-parent
              source: tests/smoke-maven-group-multidir.yaml
              version-requirement: '>1.5.12.RELEASE'
        security-advisories:
            - dependency-name: org.springframework.boot:spring-boot-starter-parent
              affected-versions:
                - < 1.5.11
              patched-versions: []
              unaffected-versions: []
        security-updates-only: true
        source:
            provider: github
            repo: dependabot/smoke-tests
            directories:
                - /maven/multi-dir/foo
                - /maven/multi-dir/bar
            commit: 56600671ab2495ed67614624740292fb5d54e4f4
    credentials:
        - host: github.com
          password: $LOCAL_GITHUB_ACCESS_TOKEN
          type: git_source
          username: x-access-token
output:
    - type: update_dependency_list
      expect:
        data:
            dependencies:
                - name: org.springframework.boot:spring-boot-starter-parent
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: pom
                      requirement: 1.5.10.RELEASE
                      source: null
                  version: 1.5.10.RELEASE
                - name: org.springframework.boot:spring-boot-starter-web
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: mysql:mysql-connector-java
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.hibernate:hibernate-java8
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.springframework.boot:spring-boot-starter-data-jpa
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.springframework.boot:spring-boot-starter-test
                  requirements:
                    - file: pom.xml
                      groups:
                        - test
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.springframework.boot:spring-boot-starter-actuator
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.springframework.boot:spring-boot-configuration-processor
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.springframework.boot:spring-boot-starter-parent
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: pom
                      requirement: 1.5.9.RELEASE
                      source: null
                  version: 1.5.9.RELEASE
                - name: org.springframework.boot:spring-boot-starter-web
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: mysql:mysql-connector-java
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.hibernate:hibernate-java8
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.springframework.boot:spring-boot-starter-data-jpa
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.springframework.boot:spring-boot-starter-test
                  requirements:
                    - file: pom.xml
                      groups:
                        - test
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.springframework.boot:spring-boot-starter-actuator
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
                - name: org.springframework.boot:spring-boot-configuration-processor
                  requirements:
                    - file: pom.xml
                      groups: []
                      metadata:
                        packaging_type: jar
                      requirement: null
                      source: null
                  version: null
            dependency_files:
                - /maven/multi-dir/foo/pom.xml
                - /maven/multi-dir/bar/pom.xml
    - type: create_pull_request
      expect:
        data:
            base-commit-sha: ""
            dependencies: []
            updated-dependency-files: []
    - type: record_update_job_error
      expect:
        data:
            error-type: unknown_error
            error-details: {}
    - type: mark_as_processed
      expect:
        data:
            base-commit-sha: 56600671ab2495ed67614624740292fb5d54e4f4
amazimbe commented 1 month ago

Resolved errors by running latest version of dependabot