fatal: repository not found

[zeke]

Hi @wei. I'm hitting a snag setting up a workflow and I'm hoping you can help diagnose.

Here's the error I'm seeing:

I've tried setting the SOURCE_REPO secret in two formats:

• https://<access_token>@github.com/github/help-docs.git
• https://zeke:<access_token>@github.com/github/help-docs.git

but either way I'm getting this output:

UPSTREAM_REPO=***
BRANCHES=master:repo-sync
remote: Repository not found.
fatal: repository 'https://github.com/github/help-docs.git/' not found

Here's the workflow file:

name: Repo Sync

on:
  schedule: 
  - cron: "0 * * * *" # every hour, on the hour

jobs:
  repo-sync:
    name: Repo Sync
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: repo-sync/github-sync@v2
      name: Sync repo to branch
      env:
        GITHUB_TOKEN: ${{ secrets.ZEKE_PAT_WITH_REPO_AND_WORKFLOW_SCOPE_FOR_REPO_SYNC }}
      with:
        source_repo: ${{ secrets.SOURCE_REPO }}
        source_branch: master
        destination_branch: repo-sync
        github_token: ${{ secrets.ZEKE_PAT_WITH_REPO_AND_WORKFLOW_SCOPE_FOR_REPO_SYNC }}
    - uses: repo-sync/pull-request@v2
      name: Create pull request
      env:
        GITHUB_TOKEN: ${{ secrets.ZEKE_PAT_WITH_REPO_AND_WORKFLOW_SCOPE_FOR_REPO_SYNC }}
      with:
        source_branch: repo-sync
        destination_branch: master
        github_token: ${{ secrets.ZEKE_PAT_WITH_REPO_AND_WORKFLOW_SCOPE_FOR_REPO_SYNC }}

The ZEKE_PAT_WITH_REPO_AND_WORKFLOW_SCOPE_FOR_REPO_SYNC secret, as its ridiculously long name suggests, is a token with repo and workflow scope created with my @zeke account, so it should have access to that repository.

Any ideas?

[wei]

I don't see anything apparent. Could you try running these two lines in a fresh environment? https://github.com/repo-sync/github-sync/blob/520596e97177727db1f2a1de14f4ded905624066/github-sync.sh#L27-L28

[spoorendonk]

I have the exact same issue. I am syncing a private repo in an organization

[wei]

Looks like GitHub actions automatically set extraheader basic authorization header in the .git/config This is overwriting the PAT provided in the url.

[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[remote "origin"]
    url = https://github.com/repo-sync/private-destination-repo
    fetch = +refs/heads/*:refs/remotes/origin/*
[gc]
    auto = 0
[http "https://github.com/"]
    extraheader = AUTHORIZATION: basic ***
[branch "master"]
    remote = origin
    merge = refs/heads/master

One way around it is to use the SSH clone url and Deploy key. The fix to HTTPS clone url support is in #21.

[wei]

@zeke @spoorendonk You can try the fix with repo-sync/github-sync@fix-private-source-with-pat

[zeke]

Thanks @wei! I will give this a try soon and get back to you.

[spoorendonk]

Works in my case. Thanks @wei

[wei]

@zeke Merging for now, let me know if you see any issues.

[zeke]

Hey sorry for the slowness following up. This fix worked for us! 🎉

[wei]

No worries!

@spoorendonk please use @v2 as the hofix branch may be deleted anytime.

[Lemmingh]

Looks like GitHub actions automatically set extraheader basic authorization header

This is introduced by Checkout V2, and is controlled by its persist-credentials option.

[connecteev]

Hi @wei

I've been stuck on the same issue. My goal: To sync all branches, tags etc from a private source repo to another private repo. I basically want to have a perfect copy of the source repo.

Attempt 1:

Here, I used the https format for source_repo. Note that I have no secrets defined in my github secrets.

My yml file in .github/workflows/repo-sync.yml (currently checked into the master branch, does it need to be on a different branch that the source_repo does NOT have?)

name: Repo Sync # from https://github.com/repo-sync/github-sync
on:
  push:
  schedule: 
  - cron: "*/3 * * * *" # every 3 mins (for now)

jobs:
  repo-sync:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
    - name: repo-sync
      uses: repo-sync/github-sync@v2
      with:
        source_repo: https://github.com/xxxxxxxxxxx/yyyyyyyyyy.git
        source_branch: '*'
        destination_branch: '*'
        github_token: ${{ secrets.GITHUB_TOKEN }}

Output of github actions run:

UPSTREAM_REPO=https://github.com/xxxxxxxxxxx/yyyyyyyyyy.git
BRANCHES=*:*
remote: Repository not found.
fatal: repository 'https://github.com/xxxxxxxxxxx/yyyyyyyyyy.git/' not found

Attempt 2:

Here, I changed the format of my source_repo to use git@ - everything else is the same. Note that I have no secrets defined in my github secrets.

My yml file in .github/workflows/repo-sync.yml (currently checked into the master branch, does it need to be on a different branch that the source_repo does NOT have?):

name: Repo Sync # from https://github.com/repo-sync/github-sync
on:
  push:
  schedule: 
  - cron: "*/3 * * * *" # every 3 mins (for now)

jobs:
  repo-sync:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
    - name: repo-sync
      uses: repo-sync/github-sync@v2
      with:
        source_repo: git@github.com:xxxxxxxxxxx/yyyyyyyyyy.git
        source_branch: '*'
        destination_branch: '*'
        github_token: ${{ secrets.GITHUB_TOKEN }}

Output of github actions run:

git@github.com:xxxxxxxxxxx/yyyyyyyyyy.git
UPSTREAM_REPO=git@github.com:xxxxxxxxxxx/yyyyyyyyyy.git
BRANCHES=*:*
Warning: Permanently added 'github.com,140.82.113.3' (RSA) to the list of known hosts.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

What am I missing?

[wei]

@connecteev See README

If source_repo is private or with another provider, either (1) use an authenticated HTTPS repo clone url like https://${access_token}@github.com/owner/repository.git or (2) set a SSH_PRIVATE_KEY secret environment variable and use the SSH clone url

[connecteev]

Thanks @wei - my question is, I do not own or have write permissions on the source repo - I just have read-only permissions. The SSH_PRIVATE_KEY has to be added to the source repo too, if I understand correctly. That would not work in my case.

Is doing this still possible and if so, how do I get the ${access_token}? Is this created on the destination repo, like from https://github.com/connecteev/xxxxx/settings/keys?

Thanks...

[wei]

@connecteev Think of it this way, if you just switched to a new computer, how would you set up authentication so you can clone the source repo?

The answer is either a deploy ssh key added to the repo (if you have permission) or using your account personal access token or ssh key.

[connecteev]

Hi @wei I went to https://github.com/settings/tokens and created a new Personal Access token with all scopes selected.

I then went to a new server and ran: git clone https://connecteev:${{ secrets.MY_PERSONAL_ACCESS_TOKEN }}@github.com/xxxxxxxxxxx/yyyyyyyyyy.git and was able to clone the repo successfully

I then changed my yml file to:

name: Repo Sync # from https://github.com/repo-sync/github-sync
on:
  push:
  schedule: 
  - cron: "*/3 * * * *" # every 3 mins (for now)

jobs:
  repo-sync:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
    - name: repo-sync
      uses: repo-sync/github-sync@v2
      with:
        source_repo: https://connecteev:${{ secrets.MY_PERSONAL_ACCESS_TOKEN }}@github.com/xxxxxxxxxxx/yyyyyyyyyy.git
        source_branch: '*'
        destination_branch: '*'
        github_token: ${{ secrets.GITHUB_TOKEN }}

but I am still seeing the same error in the output of github actions run:

UPSTREAM_REPO=https://github.com/xxxxxxxxxxx/yyyyyyyyyy.git
BRANCHES=*:*
remote: Repository not found.
fatal: repository 'https://github.com/xxxxxxxxxxx/yyyyyyyyyy.git/' not found

[wei]

@connecteev thanks for testing out the action. I just realized the @v2 tag is not pointing to the latest version @v2.1.0. It has been fixed, so if you run the action again it will not show the Repository not found. error anymore. However,

1. because you are running this action on the repo's default branch, the (push to destination repo step) will fail because GITHUB_TOKEN does not have access to overwritting GitHub action files, you could use github_token: ${{ secrets. MY_PERSONAL_ACCESS_TOKEN }} instead of secrets.GITHUB_TOKEN then the sync & push should succeed.
2. however, once it does succeed, your workflow file will be overwritten & deleted and will no longer run.

Therefore, I recommend using a third repo as a taskrunner with wei/git-sync action. It is very similar to this one but provides more flexibility allowing for any Git source and destination.

Cheers!

[connecteev]

Hi @wei thank you for the quick fix! And for the explanation. I was able to get it to work with your help 👍 Your points 1 and 2 above are spot on, however I did not need to use the wei/git-sync action (I'm sure it's useful, but I simply created another '_syncing' branch with my workflow file - of course this assumes that my source repo is never going to have a '_syncing' branch with the same name).

Here is my final yml file (both for my future reference and for someone else that may encounter the same issue).

name: Repo Sync # from https://github.com/repo-sync/github-sync

on:
  push:
  schedule:
  - cron: "*/3 * * * *" # 3 mins (for now)

jobs:
  repo-sync:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
    - name: repo-sync
      uses: repo-sync/github-sync@v2
      with:
        # MY_PERSONAL_ACCESS_TOKEN_FROM_GITHUB_SETTINGS is set to my personal access token from https://github.com/settings/tokens)
        source_repo: https://connecteev:${{ secrets.MY_PERSONAL_ACCESS_TOKEN_FROM_GITHUB_SETTINGS }}@github.com/xxxxxxxx/yyyyyyyyyy.git
        source_branch: '*'
        destination_branch: '*'
        #github_token: ${{ secrets.GITHUB_TOKEN }}
        github_token: ${{ secrets.MY_PERSONAL_ACCESS_TOKEN_FROM_GITHUB_SETTINGS }}

Really appreciate your help!

[connecteev]

Ok, I spoke a bit too soon. The sync works fine on push, but does not run every 3 minutes :( I guess because it's not on the master branch? Does that leave me with no option but to add a third repo as a taskrunner with the wei/git-sync action?

[connecteev]

@wei I'd love your thoughts on this: https://github.com/wei/git-sync/issues/18 many thanks..

[wei]

@connecteev yes. Cron only works on default branch.

[connecteev]

@wei okay thanks....so then, if I want to use Cron, repo-sync/github-sync will not work from the non-master branch. However, if I use wei/git-sync/ as a taskrunner, he cron should work if I have the yml file in the master (default) branch..and yet it does not. I'll add a comment to wei/git-sync#18 so we can continue the conversation in that repo.