dependabot could not clone private repository for swift that located same organization and granted access.

r-plus commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Package ecosystem

swift

Package manager version

5.8

Language version

5.8

Manifest location and content before the Dependabot update

`/Package.swift`

repository is appbrew/try-dependabot-spm, appbrew is the organization name.

// swift-tools-version: 5.8
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "try-dependabot-spm",
    products: [
        // Products define the executables and libraries a package produces, making them visible to other packages.
        .library(
            name: "try-dependabot-spm",
            targets: ["try-dependabot-spm"]),
    ],
    dependencies: [
        .package(url: "git@github.com:appbrew/try-dependabot-spm-private.git", exact: "0.0.1"),
    ],
    targets: [
        // Targets are the basic building blocks of a package, defining a module or a test suite.
        // Targets can depend on other targets in this package and products from dependencies.
        .target(
            name: "try-dependabot-spm"),
        .testTarget(
            name: "try-dependabot-spmTests",
            dependencies: ["try-dependabot-spm"]),
    ]
)

and Package.resolved

{
  "pins" : [
    {
      "identity" : "try-dependabot-spm-private",
      "kind" : "remoteSourceControl",
      "location" : "git@github.com:appbrew/try-dependabot-spm-private.git",
      "state" : {
        "revision" : "e645ddfa48964b5b81ab3fba2976e0989d3d176e",
        "version" : "0.0.1"
      }
    }
  ],
  "version" : 2
}

dependabot.yml content

version: 2
updates:
  - package-ecosystem: "swift"
    directory: "/"
    schedule:
      interval: "weekly"

Updated dependency

try-dependabot-spm-private repo is not yet updated from 0.0.1, so I expect dependabot will not create pull request without error.

What you expected to see, versus what you actually saw

I expect dependabot will not create pull request without error.

But dependabot will show repository clone error even if grant dependabot access to try-dependabot-spm-private repo that describing in this docs.

  proxy | 2023/09/13 15:22:36 proxy starting, commit: 93c4a893d794d736d84e940a79420e8d1180c0bd
  proxy | 2023/09/13 15:22:36 Listening (:1080)
updater | 2023-09-13T15:22:37.794093800 [721527124:main:WARN:src/devices/src/legacy/serial.rs:222] Detached the serial input due to peer close/error.
updater | time="2023-09-13T15:22:40Z" level=info msg="guest starting" commit=b073e069d366dc2f68bb8ef0134feb3c29cacadd
updater | time="2023-09-13T15:22:40Z" level=info msg="starting job..." fetcher_timeout=10m0s job_id=721527124 updater_timeout=45m0s updater_version=3ca52579b79278b4007287c479a40acdee6fb88a-swift
updater | 2023/09/13 15:22:42 INFO Raven 3.1.2 ready to catch errors
updater | 2023/09/13 15:22:43 INFO <job_721527124> Starting job processing
  proxy | 2023/09/13 15:22:44 [002] GET https://github.com:443/appbrew/try-dependabot-spm/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:44 [002] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:44 [002] 200 https://github.com:443/appbrew/try-dependabot-spm/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:44 [004] POST https://github.com:443/appbrew/try-dependabot-spm/git-upload-pack
  proxy | 2023/09/13 15:22:44 [004] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:44 [004] 200 https://github.com:443/appbrew/try-dependabot-spm/git-upload-pack
  proxy | 2023/09/13 15:22:44 [006] POST https://github.com:443/appbrew/try-dependabot-spm/git-upload-pack
  proxy | 2023/09/13 15:22:44 [006] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:44 [006] 200 https://github.com:443/appbrew/try-dependabot-spm/git-upload-pack
updater | 2023/09/13 15:22:44 INFO <job_721527124> Finished job processing
updater | time="2023-09-13T15:22:44Z" level=info msg="task complete" container_id=job-721527124-file-fetcher exit_code=0 job_id=721527124 step=fetcher
updater | 2023/09/13 15:22:46 INFO Raven 3.1.2 ready to catch errors
updater | 2023/09/13 15:22:48 INFO <job_721527124> Starting job processing
  proxy | 2023/09/13 15:22:50 [008] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [008] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:50 [008] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [008] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:50 [008] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:50 [010] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [010] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:50 [010] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [010] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:50 [010] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:50 [012] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:50 [012] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:51 [012] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [012] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:51 [012] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:51 [014] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [014] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:51 [014] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [014] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:51 [014] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:51 [016] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [016] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:51 [016] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:51 [016] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:51 [016] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:52 [018] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [018] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:52 [018] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [018] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:52 [018] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:52 [020] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [020] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:52 [020] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [020] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:52 [020] * de-auth'd request returned 401, replacing response
  proxy | 2023/09/13 15:22:52 [022] GET https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [022] * authenticating git server request (host: github.com)
  proxy | 2023/09/13 15:22:52 [022] 404 https://github.com:443/appbrew/try-dependabot-spm-private.git/info/refs?service=git-upload-pack
  proxy | 2023/09/13 15:22:52 [022] * auth'd git request returned 404, retrying without auth
  proxy | 2023/09/13 15:22:52 [022] * de-auth'd request returned 401, replacing response
updater | 2023/09/13 15:22:53 ERROR <job_721527124> Fetching git@github.com:appbrew/try-dependabot-spm-private.git
updater | warning: 'try-dependabot-spm-private': skipping cache due to an error: Failed to clone repository git@github.com:appbrew/try-dependabot-spm-private.git:
updater |     Cloning into bare repository '/home/dependabot/.cache/org.swift.swiftpm/repositories/try-dependabot-spm-private-664e0590'...
updater |     fatal: could not read Password for 'https://git@github.com': terminal prompts disabled
updater | Fetching git@github.com:appbrew/try-dependabot-spm-private.git
updater | warning: 'try-dependabot-spm-private': skipping cache due to an error: Failed to clone repository git@github.com:appbrew/try-dependabot-spm-private.git:
updater |     Cloning into bare repository '/home/dependabot/.cache/org.swift.swiftpm/repositories/try-dependabot-spm-private-664e0590'...
updater |     fatal: could not read Password for 'https://git@github.com': terminal prompts disabled
updater | error: Failed to clone repository git@github.com:appbrew/try-dependabot-spm-private.git:
updater |     Cloning into bare repository '/home/dependabot/dependabot-updater/repo/.build/repositories/try-dependabot-spm-private-664e0590'...
updater |     fatal: could not read Password for 'https://git@github.com': terminal prompts disabled
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:343:in `run_shell_command'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:38:in `formatted_deps'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:24:in `block (2 levels) in parse'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:194:in `with_git_configured'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:23:in `block in parse'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:41:in `block in in_a_temporary_repo_directory'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:41:in `chdir'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/common/lib/dependabot/shared_helpers.rb:41:in `in_a_temporary_repo_directory'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:20:in `parse'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/swift/lib/dependabot/swift/file_parser.rb:17:in `parse'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:99:in `parse_files!'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:88:in `initialize'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:21:in `new'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:21:in `create_from_job_definition'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:16:in `perform_job'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:52:in `run'
updater | 2023/09/13 15:22:53 ERROR <job_721527124> bin/update_files.rb:23:in `<main>'
updater | 2023/09/13 15:22:53 INFO <job_721527124> Sending event dc0968eb0ab240f2bc71de1549ff1a5a to Sentry
  proxy | 2023/09/13 15:22:53 [024] POST https://sentry.io:443/api/1451818/store/
  proxy | 2023/09/13 15:22:53 [024] 200 https://sentry.io:443/api/1451818/store/
updater | 2023/09/13 15:22:54 INFO <job_721527124> Finished job processing
updater | 2023/09/13 15:22:54 INFO Results:
updater | Dependabot encountered '1' error(s) during execution, please check the logs for more details.
updater | +---------------+
updater | |    Errors     |
updater | +---------------+
updater | | unknown_error |
updater | +---------------+
updater | time="2023-09-13T15:22:54Z" level=info msg="task complete" container_id=job-721527124-updater exit_code=0 job_id=721527124 step=updater

Native package manager behavior

no output without error.

$ swift package resolve
$

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

tiwoc commented 10 months ago

I'm running into the same thing. @r-plus, have you found a workaround, by any chance?

deivid-rodriguez commented 10 months ago

I think the fix for this should be a matter of replacing scp-style urls before shelling out to SwiftPM.

As a workaround, can you try replacing the url with an https url?

tiwoc commented 10 months ago

It's the same issue with https URLs, unfortunately. Dependabot just seems to ignore that we've given it access to the private repo which contains the private dependency.

deivid-rodriguez commented 10 months ago

Did you replace it in both manifest and lockfile? Can you copy redacted logs, just to see what the errors with https look like?

tiwoc commented 10 months ago

Yep, there's no trace of any non-https git URLs in our project.

Package.swift:

let package = Package(
[…]
    dependencies: [
        .package(url: "https://github.com/redacted-org/redacted-repo", exact: "0.4.8"),
[…]
    ]
[…]
)

Package.resolved:

[…]
    {
      "identity" : "redacted-repo",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/redacted-org/redacted-repo",
      "state" : {
        "revision" : "redacted-sha",
        "version" : "0.4.8"
      }
    },
[…]

Logs:

[…]
  proxy | 2023/10/23 19:46:07 [104] GET https://github.com:443/redacted-org/redacted-repo/info/refs?service=git-upload-pack
  proxy | 2023/10/23 19:46:07 [104] * authenticating git server request (host: github.com)
  proxy | 2023/10/23 19:46:07 [104] 404 https://github.com:443/redacted-org/redacted-repo/info/refs?service=git-upload-pack
  proxy | 2023/10/23 19:46:07 [104] * auth'd git request returned 404, retrying without auth
  proxy | 2023/10/23 19:46:07 [104] * de-auth'd request returned 401, replacing response
[…]
updater | error: Failed to clone repository https://github.com/redacted-org/redacted-repo:
updater |     Cloning into bare repository '/home/dependabot/dependabot-updater/repo/Packages/.build/repositories/redacted-repo-aed36908'...
updater |     fatal: could not read Username for 'https://github.com': terminal prompts disabled
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:344:in `run_shell_command'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:39:in `formatted_deps'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:25:in `block (2 levels) in parse'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:195:in `with_git_configured'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:24:in `block in parse'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:42:in `block in in_a_temporary_repo_directory'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:42:in `chdir'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/common/lib/dependabot/shared_helpers.rb:42:in `in_a_temporary_repo_directory'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser/dependency_parser.rb:21:in `parse'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/swift/lib/dependabot/swift/file_parser.rb:18:in `parse'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:99:in `parse_files!'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:90:in `initialize'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:22:in `new'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/dependency_snapshot.rb:22:in `create_from_job_definition'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:17:in `perform_job'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:53:in `run'
updater | 2023/10/23 19:48:31 ERROR <job_739621514> bin/update_files.rb:24:in `<main>'
[…]

tiwoc commented 10 months ago

Ok, there's a workaround that was helpfully suggested to me by GitHub support: with a GH access token in Dependabot secrets, we can register GH as a package registry and use that instead of relying on setting permissions in org settings (which doesn't seem to work for SPM, so this issue should still be kept open).

[…]
registries:
  github-private:
    type: git
    url: https://github.com
    username: x-access-token
    password: ${{secrets.GITHUB_TOKEN_NAME_REPLACE_ME}}

updates:
  - package-ecosystem: "swift"
    schedule:
      interval: "daily"
    registries:
      - github-private
[…]

MattSkala commented 8 months ago

Dependabot can now clone the private repository with the workaround suggested in https://github.com/dependabot/dependabot-core/issues/8027#issuecomment-1779394703, but it still fails to authenticate when downloading a package from Github Package Registry (published via KMMBridge). Is this supported? I couldn't find any related info the docs.

proxy | 2024/01/10 22:13:36 [116] 401 [https://maven.pkg.github.com:443/[...].zip](https://maven.pkg.github.com/[...].zip)

dependabot / dependabot-core