search

LoopPerfect / buckaroo

The decentralized package manager for C++ and friends 🏝️
https://buckaroo.pm
MIT License
933 stars 33 forks source link

Resolving private repositories #373

Closed roey-e closed 4 years ago

roey-e commented 4 years ago

Description

I've been using buckaroo quite some time with my private repos and I'm constantly running into the same issue. I'm using buckaroo in "SSH" mode exclusively as it easier with private repos. When using buckaroo resolve on a buckaroo project with private-repo dependencies, buckaroo results in an unresolvable error the first time, the second time it manages to resolve it and to create a lock file.

Expected Behavior

I would expect buckaroo to work the same with private repos as with public repos.

Actual Behavior

The first resolve:

info Resolve start: 2020-07-13T19:35:07
info Resolving dependencies using quick strategy...
[git] info Fetching refs from git@github.com:roey-e/six-p.git
[git] success Fetched 24 refs in 3.334
[git] info Shallow cloning git@github.com:roey-e/six-p.git
[solver] warning Unresolvable: github.com/roey-e/six-p@branch=master
info Resolve end: 2020-07-13T19:35:16
info Resolve time: 00:00:09.0553388
error The package github.com/roey-e/six-p at branch=master is unresolvable.

The second resolve:

info Resolve start: 2020-07-13T19:39:33
info Resolving dependencies using quick strategy... 
[git] info Fetching refs from git@github.com:roey-e/six-p.git
[git] success Fetched 24 refs in 2.661
[solver] info Trying github.com/roey-e/six-p at branch=master
info Resolve end: 2020-07-13T19:39:36
info Resolve time: 00:00:03.4153366
success A solution to the constraints was found. 
success The lock-file was updated.

Reading the log, I understand it has something to do with cloning the repo the first time. Deleting '~/.buckaroo' (which holds the cache if I am not mistaken) after the first resolve keeps the error.

Possible Fix

I tried to find what causes it. I debugged buckaroo and found out it might be calling sourceExplorer.FetchManifest from Solver.fs:186 which ultimately calls fetchFile of GitHubApi.fs:6. fetching a file from a private repo using 'https://raw.githubusercontent.com/' is not possible (unless using a token of course). I looked up for a way to fetch a single file using SSH, but I did not succeed (git archive --remote doesn't work with GitHub). So maybe the fix is not trivial, I'm not familiar with buckaroo's code enough to come up with a solution easily.

Steps to Reproduce

  1. Have a buckaroo project with at least one private repo as a dependency.
  2. Clone your buckaroo project.
  3. Remove the buckaroo.lock.toml file.
  4. buckaroo resolve (or buckaroo resolve --upgrade if the lock file is not up to date) -> error :(
  5. buckaroo resolve -> buckaroo.lock.toml generated :)

I used my roey-e/protobuf (branch: python-protobuf-buck) which uses my roey-e/six repo as a dependency which I privatized (roey-e/six-p).

Your Environment

nikhedonia commented 4 years ago

Thanks for looking into this! I'll schedule a debugging session tmrw

nikhedonia commented 4 years ago

Thanks @roey-e for investing time into investigating this issue. Your analysis was quite on point and was very helpful. I made a new release and you can find more context in the associated PR. Please let me know if the issue still persists in the new release