search

dependabot / dependabot-core

🤖 Dependabot's core logic for creating update PRs.
https://docs.github.com/en/code-security/dependabot
MIT License
4.56k stars 954 forks source link

Swift scanning still attempts to parse Package.swift of ignored dependencies #8753

Open samrayner opened 6 months ago

samrayner commented 6 months ago

Is there an existing issue for this?

Package ecosystem

swift

Package manager version

SPM

Language version

Swift 5.9

Manifest location and content before the Dependabot update

/Packages/Dependencies/Package.swift

import PackageDescription

let package = Package(
    name: "Dependencies",
    platforms: [.iOS(.v15)],
    products: [
        // Products define the executables and libraries a package produces, making them visible to other packages.
        .library(
            name: "Dependencies",
            targets: ["Dependencies"]
        )
    ],
    dependencies: [
        .package(url: "https://github.com/realm/realm-swift", exact: "10.40.1")
    ],
    targets: [
        // Targets are the basic building blocks of a package, defining a module or a test suite.
        // Targets can depend on other targets in this package and products from dependencies.
        .target(
            name: "Dependencies",
            dependencies: [
                .product(name: "RealmSwift", package: "realm-swift")
            ]
        )
    ]
)

dependabot.yml content

version: 2
updates:
  - package-ecosystem: "swift" # See documentation for possible values
    directory: "/Packages/Dependencies" # Location of package manifests
    schedule:
      interval: "weekly"
      day: "monday"
      time: "06:00"
    open-pull-requests-limit: 10
    groups:
      patch:
        update-types:
        - "patch"
      minor:
        update-types:
        - "minor"
    ignore:
      - dependency-name: "realm-swift"
    commit-message:
      prefix: "[dependabot] "

Updated dependency

realm-swift

What you expected to see, versus what you actually saw

We are facing an issue with realm-swift's Package.swift where it references a file that only exists on macos and not on the system Dependabot runs on (presumably Linux?).

I have opened an issue with Realm: https://github.com/realm/realm-swift/issues/8458

I had hoped in the meantime we could ignore realm-swift to skip parsing of its Package.swift but it seems that it is parsed regardless. Presumably ignore only applies to whether Dependabot creates a PR for detected updates or not?

Would it be possible for ignore to skip the parsing? It would then make it possible to use it to avoid the whole scan failing if there is an issue with a single package.

I appreciate this is an edge case and might not be feasible. Hopefully Realm will address the issue soon but I thought it would be worth bringing it to your attention anyway.

Thanks!

Native package manager behavior

No response

Images of the diff or a link to the PR, issue, or logs

Parsing of https://github.com/realm/realm-swift/blob/master/Package.swift fails:

2024-01-10T10:42:20.7718620Z Computing version for https://github.com/realm/realm-swift
2024-01-10T10:42:20.7724554Z error: Invalid manifest (compiled with: ["/opt/swift/usr/bin/swiftc", "-vfsoverlay", "/tmp/TemporaryDirectory.BBzScc/vfs.yaml", "-L", "/opt/swift/usr/lib/swift/pm/ManifestAPI", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/opt/swift/usr/lib/swift/pm/ManifestAPI", "-swift-version", "5", "-I", "/opt/swift/usr/lib/swift/pm/ManifestAPI", "-package-description-version", "5.5.0", "/Package.swift", "-Xfrontend", "-disable-implicit-concurrency-module-import", "-Xfrontend", "-disable-implicit-string-processing-module-import", "-o", "/tmp/TemporaryDirectory.ChxhXC/realm-swift-manifest"])
2024-01-10T10:42:20.7733742Z /Package.swift:106:10: warning: 'launchPath' is deprecated: renamed to 'executableURL'
2024-01-10T10:42:20.7734816Z     task.launchPath = "/usr/sbin/ioreg"
2024-01-10T10:42:20.7735435Z          ^
2024-01-10T10:42:20.7736085Z /Package.swift:106:10: note: use 'executableURL' instead
2024-01-10T10:42:20.7736871Z     task.launchPath = "/usr/sbin/ioreg"
2024-01-10T10:42:20.7737436Z          ^~~~~~~~~~
2024-01-10T10:42:20.7738213Z          executableURL
2024-01-10T10:42:20.7739098Z /Package.swift:109:10: warning: 'launch()' is deprecated: renamed to 'run'
2024-01-10T10:42:20.7739971Z     task.launch()
2024-01-10T10:42:20.7740391Z          ^
2024-01-10T10:42:20.7740945Z /Package.swift:109:10: note: use 'run' instead
2024-01-10T10:42:20.7741596Z     task.launch()
2024-01-10T10:42:20.7742014Z          ^~~~~~
2024-01-10T10:42:20.7751167Z          runFoundation/Process.swift:387: Fatal error: Error Domain=NSCocoaErrorDomain Code=260 "The file doesn’t exist."
2024-01-10T10:42:20.7753012Z 
2024-01-10T10:42:20.7753415Z *** Program crashed: Illegal instruction at 0x00007f85d9d81470 ***
2024-01-10T10:42:20.7754071Z 
2024-01-10T10:42:20.7754417Z Thread 0 "realm-swift-man" crashed:
2024-01-10T10:42:20.7754850Z 
2024-01-10T10:42:20.7755385Z 0  0x00007f85d9d81470 _assertionFailure(_:_:file:line:flags:) + 384 in libswiftCore.so
2024-01-10T10:42:20.7756182Z 
2024-01-10T10:42:20.7756328Z Thread 1:
2024-01-10T10:42:20.7756578Z 
2024-01-10T10:42:20.7756805Z 0  0x00007f85d8de1ade <unknown> in libc.so.6
2024-01-10T10:42:20.7757289Z 
2024-01-10T10:42:20.7757296Z 
2024-01-10T10:42:20.7757442Z Registers:
2024-01-10T10:42:20.7757687Z 
2024-01-10T10:42:20.7758431Z rax 0x0000000200000003  07 07 07 07 07 07 07 07 07 07 07 07 07 07 07 07  ················
2024-01-10T10:42:20.7760044Z rdx 0x000055d270f487d0  58 ed 17 da 85 7f 00 00 03 00 00 00 00 00 00 00  Xí·Ú············
2024-01-10T10:42:20.7761359Z rcx 0xfffffffe00000000  58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58  XXXXXXXXXXXXXXXX
2024-01-10T10:42:20.7762663Z rbx 0x0000000000000003  58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58  XXXXXXXXXXXXXXXX
2024-01-10T10:42:20.7764099Z rsi 0x000055d270f487d8  03 00 00 00 00 00 00 00 90 00 00 00 00 00 00 80  ················
2024-01-10T10:42:20.7765611Z rdi 0x000055d270f36010  02 00 02 00 01 00 01 00 01 00 02 00 01 00 00 00  ················
2024-01-10T10:42:20.7767009Z rbp 0x0000000000000044  02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02  ················
2024-01-10T10:42:20.7768436Z rsp 0x00007ffcd25baab0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ················
2024-01-10T10:42:20.7769884Z  r8 0x000055d270f4aa20  4a 0f 27 5d 05 00 00 00 fd e7 a3 2e 90 25 fb c8  J·']····ýç£.·%ûÈ
2024-01-10T10:42:20.7771161Z  r9 0x00007ffcd25ba800  46 61 74 61 6c 20 65 72 72 6f 72 3a 20 45 72 72  Fatal error: Err
2024-01-10T10:42:20.7772424Z r10 0x0000000000000000  46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46  FFFFFFFFFFFFFFFF
2024-01-10T10:42:20.7773701Z r11 0xc8fb25902ea3e7fd  46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46  FFFFFFFFFFFFFFFF
2024-01-10T10:42:20.7775011Z r12 0x0000000000000000  46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46  FFFFFFFFFFFFFFFF
2024-01-10T10:42:20.7776524Z r13 0x0000000000000002  46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46  FFFFFFFFFFFFFFFF
2024-01-10T10:42:20.7778147Z r14 0x00007f85d981e571  46 61 74 61 6c 20 65 72 72 6f 72 00 5b 36 66 5d  Fatal error·[6f]
2024-01-10T10:42:20.7779424Z r15 0x000000000000000b  46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46  FFFFFFFFFFFFFFFF
2024-01-10T10:42:20.7780866Z rip 0x00007f85d9d81470  0f 0b 48 83 ec 08 48 8d 05 b3 60 41 00 48 8d 3d  ··H·ì·H··³`A·H·=
2024-01-10T10:42:20.7781637Z 
2024-01-10T10:42:20.7781842Z rflags 0x0000000000010246  ZF PF
2024-01-10T10:42:20.7782210Z 
2024-01-10T10:42:20.7782411Z cs 0x0033  fs 0x0000  gs 0x0000
2024-01-10T10:42:20.7782775Z 
2024-01-10T10:42:20.7782782Z 
2024-01-10T10:42:20.7782964Z Images (18 omitted):
2024-01-10T10:42:20.7783248Z 
2024-01-10T10:42:20.7797807Z 0x00007f85d8cc9000–0x00007f85d8e85141 203de0ae33b53fee1578b117cb4123e85d0534f0 libc.so.6       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-01-10T10:42:20.7800696Z 0x00007f85d9c1e000–0x00007f85da15bd68 <no build ID>                            libswiftCore.so /opt/swift/usr/lib/swift/linux/libswiftCore.so in https://github.com/realm/realm-swift
2024-01-10T10:42:20.7803120Z updater | 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/common/lib/dependabot/shared_helpers.rb:427:in `run_shell_command'
2024-01-10T10:42:20.7806634Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11142/lib/types/private/methods/call_validation.rb:153:in `bind_call'
2024-01-10T10:42:20.7810260Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11142/lib/types/private/methods/call_validation.rb:153:in `validate_call_skip_block_type'
2024-01-10T10:42:20.7814153Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11142/lib/types/private/methods/call_validation.rb:95:in `block in create_validator_slow_skip_block_type'
2024-01-10T10:42:20.7818110Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:39:in `formatted_deps'
2024-01-10T10:42:20.7820535Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:25:in `block (2 levels) in parse'
2024-01-10T10:42:20.7822815Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/common/lib/dependabot/shared_helpers.rb:264:in `with_git_configured'
2024-01-10T10:42:20.7825947Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11142/lib/types/private/methods/call_validation.rb:256:in `bind_call'
2024-01-10T10:42:20.7829383Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11142/lib/types/private/methods/call_validation.rb:256:in `validate_call'
2024-01-10T10:42:20.7832967Z updater | 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11142/lib/types/private/methods/_methods.rb:275:in `block in _on_method_added'
2024-01-10T10:42:20.7835858Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:24:in `block in parse'
2024-01-10T10:42:20.7838181Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/common/lib/dependabot/shared_helpers.rb:57:in `block in in_a_temporary_repo_directory'
2024-01-10T10:42:20.7840242Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/common/lib/dependabot/shared_helpers.rb:57:in `chdir'
2024-01-10T10:42:20.7842235Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/common/lib/dependabot/shared_helpers.rb:57:in `in_a_temporary_repo_directory'
2024-01-10T10:42:20.7845590Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11142/lib/types/private/methods/call_validation.rb:256:in `bind_call'
2024-01-10T10:42:20.7849238Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11142/lib/types/private/methods/call_validation.rb:256:in `validate_call'
2024-01-10T10:42:20.7852749Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11142/lib/types/private/methods/_methods.rb:275:in `block in _on_method_added'
2024-01-10T10:42:20.7855482Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:21:in `parse'
2024-01-10T10:42:20.7857488Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/swift/lib/dependabot/swift/file_parser.rb:18:in `parse'
2024-01-10T10:42:20.7859849Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:101:in `parse_files!'
2024-01-10T10:42:20.7862237Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:92:in `initialize'
2024-01-10T10:42:20.7864457Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:24:in `new'
2024-01-10T10:42:20.7866854Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:24:in `create_from_job_definition'
2024-01-10T10:42:20.7869340Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:21:in `perform_job'
2024-01-10T10:42:20.7871524Z 2024/01/10 10:42:20 ERROR <job_772374558> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:53:in `run'
2024-01-10T10:42:20.7873024Z 2024/01/10 10:42:20 ERROR <job_772374558> bin/update_files.rb:24:in `<main>'
2024-01-10T10:42:21.6894936Z   proxy | 2024/01/10 10:42:21 [332] POST https://dependabot-actions.githubapp.com:443/update_jobs/772374558/record_update_job_error
2024-01-10T10:42:21.7821607Z   proxy | 2024/01/10 10:42:21 [332] 204 https://dependabot-actions.githubapp.com:443/update_jobs/772374558/record_update_job_error
2024-01-10T10:42:21.7944491Z   proxy | 2024/01/10 10:42:21 [334] POST https://dependabot-actions.githubapp.com:443/update_jobs/772374558/record_update_job_unknown_error
2024-01-10T10:42:21.8296306Z   proxy | 2024/01/10 10:42:21 [334] 204 https://dependabot-actions.githubapp.com:443/update_jobs/772374558/record_update_job_unknown_error
2024-01-10T10:42:21.8791993Z   proxy | 2024/01/10 10:42:21 [336] PATCH https://dependabot-actions.githubapp.com:443/update_jobs/772374558/mark_as_processed
2024-01-10T10:42:21.9286271Z   proxy | 2024/01/10 10:42:21 [336] 204 https://dependabot-actions.githubapp.com:443/update_jobs/772374558/mark_as_processed
2024-01-10T10:42:21.9300109Z updater | 2024/01/10 10:42:21 INFO <job_772374558> Finished job processing
2024-01-10T10:42:21.9313701Z updater | 2024/01/10 10:42:21 INFO Results:
2024-01-10T10:42:21.9316407Z Dependabot encountered '1' error(s) during execution, please check the logs for more details.
2024-01-10T10:42:21.9318708Z +--------------------+
2024-01-10T10:42:21.9323998Z |       Errors       |
2024-01-10T10:42:21.9326607Z +--------------------+
2024-01-10T10:42:21.9327590Z | update_files_error |
2024-01-10T10:42:21.9328583Z +--------------------+
2024-01-10T10:42:24.2646816Z Failure running container 414e030d4861834743b771db6f686e8c0f4baee1e4ba17be7eab5fb8e945e185
2024-01-10T10:42:26.1034498Z Cleaned up container 414e030d4861834743b771db6f686e8c0f4baee1e4ba17be7eab5fb8e945e185
2024-01-10T10:42:26.7358839Z (node:1597) [DEP0147] DeprecationWarning: In future versions of Node.js, fs.rmdir(path, { recursive: true }) will be removed. Use fs.rm(path, { recursive: true }) instead
2024-01-10T10:42:26.7361090Z (Use `node --trace-deprecation ...` to show where the warning was created)
2024-01-10T10:42:26.8377802Z ##[error]Dependabot encountered an error performing the update

Error: The updater encountered one or more errors.

For more information see: https://github.com/waveremit/remit-ios/network/updates/772374558 (write access to the repository is required to view the log)
2024-01-10T10:42:26.8416323Z 🤖 ~ finished: error reported to Dependabot ~

Smallest manifest that reproduces the issue

See above :)

deivid-rodriguez commented 6 months ago

In general, ignore just means "don't create PRs for this dependency", but we normally can't ensure the dependency won't be parsed.

This is because:

To sum up, I don't see an easy way of addressing this.

samrayner commented 6 months ago

Thanks @deivid-rodriguez ! Understood. It's sounding promising that Realm will fix this.