Can not resolve dependency's dependency when mirror it from url to package identifier

[hollyoops]

Description

My package has a dependency package (RecoilSwift), and the dependency has one dependency package as well. My package uses the package identifier to pull dependencies.

My Package.swift :

let package = Package(
    name: "Dependencies",
   ...
    products: [
        .library( name: "Dependencies", targets: ["Dependencies"]),
    ],
    dependencies: [
        .package(id: "hollyoops.RecoilSwift", from: "0.2.1"),
    ],
    targets: [
        .target(
            name: "Dependencies",
            dependencies: [
                .product(name: "RecoilSwift", package: "hollyoops.RecoilSwift"),
            ])
    ]
)

RecoilSwift's Package.swift, RecoilSwift has a dependency SwiftUI-Hooks, RecoilSwift use URL to fetch the dependencies:

let package = Package(
    name: "RecoilSwift",
    ...
    dependencies: [
      .package(url: "https://github.com/hollyoops/SwiftUI-Hooks", from: "0.0.3")
    ],
    targets: [
        .target(
            name: "RecoilSwift",
            dependencies: [
              .product(name: "Hooks", package: "SwiftUI-Hooks")
            ],
            path: "Sources")
    ]
)

My goal is to let the SwiftUI-Hooks use the swiftPM registry to fetch all the code. So I use the set mirror subcommand to "mapping" the URL to package id. Here is the mirrors.json:

{
  "object": [
    {
      "mirror": "hollyoops.SwiftUI-Hooks",
      "original": "https://github.com/hollyoops/SwiftUI-Hooks"
    }
  ],
  "version": 1
}

However, when we resolve the dependencies we get an error:

warning: dependency 'hollyoops.RecoilSwift' is missing; downloading again error: 'hollyoops.RecoilSwift': unknown package 'SwiftUI-Hooks' in dependencies of target 'RecoilSwift'; valid packages are: 'hollyoops.SwiftUI-Hooks'

How can I resolve the dependency's dependency without changing the source code of the package while fetching by package identifier?

Expected behavior

Can resolve the dependency's dependency correctly

Actual behavior

get an error:

error: 'hollyoops.RecoilSwift': unknown package 'SwiftUI-Hooks' in dependencies of target 'RecoilSwift'; valid packages are: 'hollyoops.SwiftUI-Hooks'

Steps to reproduce

1. Add a dependency that has some dependencies. and the dependencies fetched by the URL
2. Add a mirror to map the dependencies from the URL to the package-identifier
3. resolve the package swift package reslove

Swift Package Manager version/commit hash

5.7.1

Swift & OS version (output of swift --version && uname -a)

swift-driver version: 1.62.15 Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51) Target: arm64-apple-macosx13.0 Darwin XXX.lan 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:15:09 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T6000 arm64

[tomerd]

thanks for reporting @hollyoops the current mirroring solution is not designed to help with mapping source control URLs to registry identities. we could look into making that possible.

one workaround to make this work is to have the registry host hollyoops.SwiftUI-Hooks and make sure it has the metadata to map it back to https://github.com/hollyoops/SwiftUI-Hooks. then in the main project add a dependency on hollyoops.SwiftUI-Hooks as well which will force it to use the registry version. when SwiftPM resolves the dependencies it will query the registry for hollyoops.SwiftUI-Hooks and since https://github.com/hollyoops/SwiftUI-Hooks is associated with it will deduplicate the two and prefer the registry version.

[hollyoops]

thanks for reporting @hollyoops the current mirroring solution is not designed to help with mapping source control URLs to registry identities. we could look into making that possible.

one workaround to make this work is to have the registry host hollyoops.SwiftUI-Hooks and make sure it has the metadata to map it back to https://github.com/hollyoops/SwiftUI-Hooks. then in the main project add a dependency on hollyoops.SwiftUI-Hooks as well which will force it to use the registry version. when SwiftPM resolves the dependencies it will query the registry for hollyoops.SwiftUI-Hooks and since https://github.com/hollyoops/SwiftUI-Hooks is associated with it will deduplicate the two and prefer the registry version.

The metadata? Do you mean the release metadata in the registry server's response?

{
  "id": "mona.LinkedList",
  "version": "1.1.1",
  "resources": [ ... ],
  "metadata": {
    "@context": ["http://schema.org/"],
    "@type": "SoftwareSourceCode",
    "name": "LinkedList",
    "description": "One thing links to another.",
    "keywords": ["data-structure", "collection"],
    "version": "1.1.1",
    "codeRepository": "https://github.com/mona/LinkedList",
    "license": "https://www.apache.org/licenses/LICENSE-2.0",
    "programmingLanguage": {
      "@type": "ComputerLanguage",
      "name": "Swift",
      "url": "https://swift.org"
    },
    "author": {
        "@type": "Person",
        "@id": "https://example.com/mona",
        "givenName": "Mona",
        "middleName": "Lisa",
        "familyName": "Octocat"
    }
  }
}

[tomerd]

https://github.com/apple/swift-package-manager/blob/main/Documentation/Registry.md#41-list-package-releases

see information about canonical and alternative Link headers:

A server MAY include a Link entry with the canonical relation type that locates the source repository of the package.

A server MAY include one or more Link entries with the alternate relation type for other source repository locations.

[tomerd]

btw, not sure which registry you are using, but the artifactory based one has this implemented via their metadata system

[hollyoops]

https://github.com/apple/swift-package-manager/blob/main/Documentation/Registry.md#41-list-package-releases

see information about canonical and alternative Link headers:

A server MAY include a Link entry with the canonical relation type that locates the source repository of the package.

A server MAY include one or more Link entries with the alternate relation type for other source repository locations.

I tried your solution. What I did are:

1. Add the canonical and alternative in the Link header

Link: <https://spm-registry-service.onrender.com/hollyoops/SwiftUI-Hooks/0.0.3>; rel="latest-version",<https://github.com/hollyoops/SwiftUI-Hooks>; rel="canonical",<ssh://git@github.com:hollyoops/SwiftUI-Hooks.git>; rel="alternate"

2. Add hollyoops.SwiftUI-Hooks as a dependency on the main project

let package = Package(
    ...
    dependencies: [
        .package(id: "hollyoops.SwiftUI-Hooks", from: "0.0.3"),
        .package(id: "hollyoops.RecoilSwift", from: "0.2.1")
    ],
    targets: [
        .target(
            name: "Dependencies",
            dependencies: [
                .product(name: "RecoilSwift", package: "hollyoops.RecoilSwift"),
                .product(name: "Hooks", package: "hollyoops.SwiftUI-Hooks"),
            ])
            ...
    ]
)

With no luck. SPM just resolved two version of SwiftUI-Hooks. it dose not deduplicate the two .

[tomerd]

thanks for the additional info @hollyoops I see you are using this in Xcode which still not fully supported yet. you should have more luck on the command line since you need to pass another argument to SwiftPM to activate the URL to registry identity mapping. this will be the default in the upcoming release, but needs an explicit flag right now:

  --replace-scm-with-registry
                          look up source control dependencies in the registry and use the registry to retrieve them
                          instead of source control when possible (default: disabled)

[hollyoops]

thanks for the additional info @hollyoops I see you are using this in Xcode which still not fully supported yet. you should have more luck on the command line since you need to pass another argument to SwiftPM to activate the URL to registry identity mapping. this will be the default in the upcoming release, but needs an explicit flag right now:

  --replace-scm-with-registry
                          look up source control dependencies in the registry and use the registry to retrieve them
                          instead of source control when possible (default: disabled)

Thank you!! It works with the command line! swift package --replace-scm-with-registry resolve resolve the package successfully.

But however, after that, when I open the Xcode. Xcode still doesn't know where are the packages and still tries to fetch packages from Github. I guess this is the Xcode issue? It does not fully support it. Do you know when Xcode can support this?

[tomerd]

@hollyoops yes Xcode support is still work in progress. We cannot speak to Xcode release schedule unfortunately.

[tomerd]

also: https://github.com/apple/swift-package-manager/pull/6017

[hollyoops]

I tested on Swift 5.9. Looks like the mirror works.

SPM originally possessed numerous powerful features and parameters. However, Xcode does not provide a user interface to customize parameters which supported by SPM during the package resolution process. This greatly restricts the potential of SPM.