[FEATURE] private repository support

[peterclemenko]

Adding the ability to easily load from a private repository might help with managing content purchased from places like booth and gumroad

[anatawa12]

There already is feature to authenticate HTTP request with custom headers.

https://vcc.docs.vrchat.com/guides/community-repositories/#providing-request-headers

Some repositories may require you to provide additional headers with your request. For example, if you want to access a private repository, you may need to provide an authorization token.

[Dreadrith]

Unfortunately, that's not enough from what I've tried unless I'm missing something. While yes, you can add a private repository's .json listing to VCC, it doesn't seem capable of fetching a package in that private repository. I've done a test to confirm what I'm saying.

I've made the repository and the listing in the proper format. VCC can fetch the listing when given the required authorization header.

It's listed and displayed in VCC correctly.

When attempting to download the package, it runs into an error. My only assumption is that it attempts to get the .zip from the URL without regards to the authorization given to the listing.

I've tried looking around on Github's API, Stackoverflow, and Google for a way to be able to fetch the asset with a GET parameter but I haven't found any easy way. For my private repositories, I had to make a cloud function that is public that fetches the .zip for you.

[anatawa12]

As far as I know, the VCC/VPM will download the package zip with headers of vpm repository so if your http server (in your case, github private repository) can authenticate with some http header to get json, it should work as expected.

[anatawa12]

IMPORTANT: DO NOT USE THIS METHOD IN PRODUCTION This proves the feature for downloading assets from private vpm repository is available but This requires make your GitHub token public which is extremely bad for safety In other words, do not use private github repository for private vpm repository. DO NOT USE THIS METHOD IN PRODUCTION

According to github api docs, getting https://api.github.com/repos/<owner>/<repo>/releases/assets/<assetid> with Accept: application/octet-stream will redirects to file download url.

To download the asset's binary content, set the Accept header of the request to application/octet-stream. The API will either redirect the client to the location, or stream it directly if possible. API clients should handle both a 200 or 302 response.

I created my own private test repository and I tested that this works with curl.

curl -vsL  -H "Accept: application/octet-stream" -H "Authorization: Bearer $TOKEN" --url "https://api.github.com/repos/anatawa12/private-vpm-test/releases/assets/159712559" > downloaded.zip

I just confirmed we can use this to download assets from github private repository.

Example repo: https://gist.githubusercontent.com/anatawa12/d9f5305b46ed49ac807882b3dd293d4b/raw/vpm.json (this requires PAT so you cannot actually use this)

This requires publishing the PAT to public which is not good so I think people should create proxy server with authentication.

(Even with some stricter token such as fine-grained PAT or github app token, people can download public repository as you.)

[Dreadrith]

Thank you for the info. I didn't know you can get the file directly by providing that accept header. I guess I missed it. Since Github is used as a reference often, I think this info should be provided in the docs as otherwise the existing docs aren't sufficient and may mislead. The VCC already requires you to use a token to get the listing. Otherwise, as you mentioned, a proxy server is the only way to get both the listing and the package.

[anatawa12]

As I mentioned above, I think using private github repository as a private vpm repository is not good because it requires leaking github token (as I mentioned above) so I don't think this should be documented.

EDIT: if github terms allows, we may add buyer to the private repository as a some kind of collaborator and use their own PAT for downloading from github private repository instead of leaking PAT. However I think documenting this will cause leaking PAT so I think this should not be documented.

[anatawa12]

I think we (or VRChat-community) should create software specific for private vpm repository.

I had had idea for this but I don't have enough time and enough motivation so I don't create that yet. I personally think it might be good to use cloudflare workers + R2 + D1 because it's extremely cheap for small-scale use case.

[orels1]

While I do not have anything to share on this feature-wise, creating a custom listing backend that can handle access control is the way to go at this point.

VRLabs public repository is an interesting example of a dynamic backend that can serve a unique set of packages via a simple unique id