Allow a "git" registry to be consumed from a repository subdirectory

[demianmnave-pti]

---

Currently, vcpkg assumes that a registry is its own separately versioned project and that its content directories live at the root of the repository. It would be convenient to also allow hosting a custom port directly from a project subdirectory as an alternative to bundling multiple ports together under a single custom registry.

Proposed solution Here is an example vcpkg-configuration.json showing the single "path" entry needed to support this registry model. Supporting the "path" field does not change the semantics or configuration of the current registry model, only that of the "git" registry entry in vcpkg-configuration.json.

The repository and branch reference show below are live as of Sep 14, 2022:

{
  "default-registry": {
    "$info": "The currently support vcpkg baseline",
    "kind": "builtin",
    "baseline": "d474b26040207c89d81f50dd39be7d335f1d3745"
  },

  "registries": [
    {
      "$info": "A custom git registry published as a relative subdirectory of a project",
      "kind": "git",
      "repository": "https://github.com/propulsortech/lexertl14",
      "path": "vcpkg",
      "packages": [ "lexertl14" ],
      "baseline": "5dbc61f0ced9aef6d75f8c9025e7a9366137a94d",
      "reference": "1-experiment-with-lexertl14-as-its-own-vcpkg-registry"
    }
  ]
}

Describe alternatives you've considered At the moment, I have a single custom registry containing ports for several projects. The other alternatives I have considered include exposing the ports and versions directories in the root of the project and creating a separate project under the same namespace just for the registry (e.g. propulsortech/lexertl14-vcpkg). Both options work as expected but are not as elegant or straightforward to maintain as having the custom registry under the project itself.

Additional context I dove into the vcpkg-tool source (at 3c502fa609 on main) to convince myself that this feature is even possible. The files affected appear to be vcpkg/configuration.cpp and vcpkg/registries.cpp, as well as vcpkg-test/registries.cpp for verifying that parsing works and azure-pipelines/end-to-end-tests-dir/registries.ps1 for e2e testing. Documentation for the git registry would also need to be updated to include a description of how the "path" field works.

[demianmnave-pti]

I was able to implement enough of the "path" field support so that the JSON example from the issue description works with a local build of vcpkg.exe. For reference, I also made a fork to record my changes for future reference: https://github.com/demianmnave-pti/vcpkg-tool.git

[JackBoosY]

Currently vcpkg's integrated manifest configuration file (vcpkg.json) is solution-specific rather than per-project. Since manifest implements version isolation, I think we can implement this feature in the future. @BillyONeal what do you think about this?

[demianmnave-pti]

@JackBoosY Thanks for the input. I'm wondering how you envision manifests, version isolation, and vcpkg.json helping here?

I think what I'm suggesting is simpler than what you have in mind. I just want to expose a custom git registry under a subdirectory of a project, rather than have the registry "version" and "ports" paths exposed in the root of the project.

This isn't adding new functionality, it's just a small addition to the supported fields in vcpkg-configuration.json. You can see the limited extent of the changes if you'd like to take a look at my fork.

Also, if it helps clarify my request, the initial registries specification mentions what would appear to be the functionality I am after (copied here for reference, emphasis mine):

• <registry-implementation.git>:
  • "kind": The string "git"
  • "repository": A URI
  • Optionally, "path": An absolute path into the git repository
  • Optionally, "ref": A git reference

I did not find any information in the issues or in the documentation concerning why the "path" field was dropped from the current git registry specification. If you happen to know the reason or if you have a reference to a public discussion, please do let me know.

Thank you.

[BillyONeal]

Ah, I think this was interpreted as 'allow the registry itself to have a path relative to a vcpkg.json' rather than 'implement the path field'.

[demianmnave-pti]

My apologies for being unclear at the outset. Hopefully the request is clear now, though.

[JackBoosY]

Sorry I misunderstood what you meant.

[demianmnave-pti]

No worries, thanks for considering my request.

If you'd like to move forward with this feature, I can work on a pull request. Just let me know.

[demianmnave-pti]

I also just discovered the concept of an orphaned git branch, which would provide a viable alternative. The only drawback is that a custom registry in an orphaned branch is not as transparent as one in a subdirectory.

I understand if you would prefer to table my feature request because similar functionality exists.

[BillyONeal]

I also just discovered the concept of an orphaned git branch, which would provide a viable alternative.

If it's orphaned then the commits associated with be removed at the next git gc as soon as they age out of the reflog.

[demianmnave-pti]

@BillyONeal I thought git gc only prunes dangling or unreachable commits? The commits in an orphaned branch are reachable from that branch, so I think they would be safe from git gc.

Just as an experiment, I tried git gc --prune=now on a clone with an orphan branch, and all commits in the branch seem intact. Please do correct me if I'm missing something, though.

[BillyONeal]

Hmmm I think I may be mixing up orphan and detached...

[demianmnave-pti]

No worries!

[demianmnave-pti]

Fork moved here: https://github.com/propulsortech/vcpkg-tool/tree/support-a-git-registry-under-a-subdirectory

[demianmnave-pti]

This has dropped in priority for us, so I've moved the fork back to: https://github.com/demianmnave-pti/vcpkg-tool.git

[github-actions[bot]]

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

[agriyakhetarpal]

Hi, I see that this was marked as a stale issue and closed by the bot as not planned – we would love this functionality to be included as a part of vcpkg since we use multiple custom registries in different repositories when building our CMake target on one repository. As a workaround, I am considering using different branches in the same repository which mirror the registries as mentioned above; if there is a better way at this time (December 2023), any help would be appreciated!

[github-actions[bot]]

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

[agriyakhetarpal]

Hi, it would be great to keep this feature request open