search

AkashRajpurohit / git-sync

🔄 A simple tool to backup and sync your git repositories
https://github.com/AkashRajpurohit/git-sync/wiki
MIT License
159 stars 8 forks source link

Error while cloning repositories #29

Open aeonSolutions opened 3 months ago

aeonSolutions commented 3 months ago

Describe the Bug

see log below

Steps to reproduce

executed the exe file

Relevant log output

{"level":"fatal","time":"2024-08-22T09:05:22.567+0100","message":"Error cloning repo aeonSolutions/AeonLabs-AeonLabs-HomeAutomation-Distributed-power_storage-Selectable-USB-DC-power-outlet: exit status 128\n","stacktrace":"github.com/AkashRajpurohit/git-sync/pkg/logger.Fatalf\n\t/home/runner/work/git-sync/git-sync/pkg/logger/logger.go:96\ngithub.com/AkashRajpurohit/git-sync/pkg/sync.CloneOrUpdateRepo\n\t/home/runner/work/git-sync/git-sync/pkg/sync/sync.go:28\ngithub.com/AkashRajpurohit/git-sync/pkg/github.GitHubClient.Sync.func1\n\t/home/runner/work/git-sync/git-sync/pkg/github/github.go:51"}

Which browser are you using? (if relevant)

no relevant

AkashRajpurohit commented 3 months ago

Seems like it is not able to clone the repository, can you check for few things

  1. Is this a private repository?
  2. Are you able to clone the repository directly using the git command?
  3. Does the access token have correct permissions?
aeonSolutions commented 3 months ago

Is this a private repository? No. Are you able to clone the repository directly using the git command? Yes. Does the access token have correct permissions? Yes

AkashRajpurohit commented 3 months ago

The current set of debug logs won't be helpful, can you download the v0.8.2 version and run the git-sync command again with log level set to debug --log-level=debug

aeonSolutions commented 3 months ago

ok. so i did what you asked. here's output where the errors occur:

{"level":"info","time":"2024-08-23T07:55:10.831+0100","message":"Updating repo: aeonSolutions/-openScienceResearch-12bit-LCD-1.69-TFT-Smart-DAQ-Device-with-unique-data-fingerprint"}
{"level":"debug","time":"2024-08-23T07:55:11.014+0100","message":"Output: fatal: not a git repository: '\\aeonSolutions\\WebCoder-Multi-Platform-Framework\\WebCoder-Multi-Platform-Framework.git'\n\n"}
{"level":"fatal","time":"2024-08-23T07:55:11.017+0100","message":"Error updating repo aeonSolutions/WebCoder-Multi-Platform-Framework: exit status 128\n","stacktrace":"github.com/AkashRajpurohit/git-sync/pkg/logger.Fatalf\n\t/home/runner/work/git-sync/git-sync/pkg/logger/logger.go:96\ngithub.com/AkashRajpurohit/git-sync/pkg/sync.CloneOrUpdateRepo\n\t/home/runner/work/git-sync/git-sync/pkg/sync/sync.go:39\ngithub.com/AkashRajpurohit/git-sync/pkg/github.GitHubClient.Sync.func1\n\t/home/runner/work/git-sync/git-sync/pkg/github/github.go:51"}
AkashRajpurohit commented 3 months ago

This is a Windows system, right? I never tested git-sync on Windows, so I might have to do it if I get a chance. Meanwhile, can you share your config file (excluding the access token details)?

aeonSolutions commented 3 months ago

Yes. I'm running windows 11. My config file :

backup_dir: \
cron: ""
exclude_orgs: []
exclude_repos: []
include_forks: false
include_orgs: []
include_repos: []
include_wiki: true
platform: github
server:
    domain: github.com
    protocol: https
token: ""
username: ""
workspace: ""
AkashRajpurohit commented 3 months ago

Can you update the backup_dir to be a valid path on your file system and re-run the program once?

aeonSolutions commented 3 months ago

the backup path is not the issue. As it created folders and subfolder in the root of the disk drive

AkashRajpurohit commented 3 months ago

Can you navigate to this path \\aeonSolutions\\WebCoder-Multi-Platform-Framework\\WebCoder-Multi-Platform-Framework.git and do git status or git log to see what the output is? Strange that from the logs it shows that it is not a git repository.

aeonSolutions commented 3 months ago

that repository is located here: https://github.com/aeonSolutions/WebCoder-Multi-Platform-Framework

here's the output image

AkashRajpurohit commented 3 months ago

Not this, the path specified in the log shows \\aeonSolutions\\WebCoder-Multi-Platform-Framework\\WebCoder-Multi-Platform-Framework.git, can you cd into this path and then type git status command.

aeonSolutions commented 3 months ago

here it is

image

AkashRajpurohit commented 3 months ago

Yeah so git-sync is throwing back the same error seen here, what are the contents of this folder, if you do ls -al you can see the contents, is it a bare repository?

aeonSolutions commented 3 months ago

i did a git-sync again and now the error is on another repository image

aeonSolutions commented 3 months ago

Here's the output of "dir /s"

D:\aeonSolutions\WebCoder-Multi-Platform-Framework>dir /s
 Volume in drive D is New Volume
 Volume Serial Number is D8FA-B0D9

 Directory of D:\aeonSolutions\WebCoder-Multi-Platform-Framework

08/23/2024  03:10 PM    <DIR>          .
08/23/2024  03:10 PM    <DIR>          ..
08/23/2024  03:10 PM    <DIR>          WebCoder-Multi-Platform-Framework.git
               0 File(s)              0 bytes

 Directory of D:\aeonSolutions\WebCoder-Multi-Platform-Framework\WebCoder-Multi-Platform-Framework.git

08/23/2024  03:10 PM    <DIR>          .
08/23/2024  03:10 PM    <DIR>          ..
08/23/2024  03:10 PM               260 config
08/23/2024  03:10 PM                73 description
08/23/2024  03:10 PM    <DIR>          hooks
08/23/2024  03:10 PM    <DIR>          info
08/23/2024  03:10 PM    <DIR>          objects
               2 File(s)            333 bytes

 Directory of D:\aeonSolutions\WebCoder-Multi-Platform-Framework\WebCoder-Multi-Platform-Framework.git\hooks

08/23/2024  03:10 PM    <DIR>          .
08/23/2024  03:10 PM    <DIR>          ..
08/23/2024  03:10 PM               478 applypatch-msg.sample
08/23/2024  03:10 PM               896 commit-msg.sample
08/23/2024  03:10 PM             4,726 fsmonitor-watchman.sample
08/23/2024  03:10 PM               189 post-update.sample
08/23/2024  03:10 PM               424 pre-applypatch.sample
08/23/2024  03:10 PM             1,649 pre-commit.sample
08/23/2024  03:10 PM               416 pre-merge-commit.sample
08/23/2024  03:10 PM             1,374 pre-push.sample
08/23/2024  03:10 PM             4,898 pre-rebase.sample
08/23/2024  03:10 PM               544 pre-receive.sample
08/23/2024  03:10 PM             1,492 prepare-commit-msg.sample
08/23/2024  03:10 PM             2,783 push-to-checkout.sample
08/23/2024  03:10 PM             2,308 sendemail-validate.sample
08/23/2024  03:10 PM             3,650 update.sample
              14 File(s)         25,827 bytes

 Directory of D:\aeonSolutions\WebCoder-Multi-Platform-Framework\WebCoder-Multi-Platform-Framework.git\info

08/23/2024  03:10 PM    <DIR>          .
08/23/2024  03:10 PM    <DIR>          ..
08/23/2024  03:10 PM               240 exclude
               1 File(s)            240 bytes

 Directory of D:\aeonSolutions\WebCoder-Multi-Platform-Framework\WebCoder-Multi-Platform-Framework.git\objects

08/23/2024  03:10 PM    <DIR>          .
08/23/2024  03:10 PM    <DIR>          ..
08/23/2024  03:10 PM    <DIR>          info
08/23/2024  03:10 PM    <DIR>          pack
               0 File(s)              0 bytes

 Directory of D:\aeonSolutions\WebCoder-Multi-Platform-Framework\WebCoder-Multi-Platform-Framework.git\objects\info

08/23/2024  03:10 PM    <DIR>          .
08/23/2024  03:10 PM    <DIR>          ..
               0 File(s)              0 bytes

 Directory of D:\aeonSolutions\WebCoder-Multi-Platform-Framework\WebCoder-Multi-Platform-Framework.git\objects\pack

08/23/2024  03:10 PM    <DIR>          .
08/23/2024  03:10 PM    <DIR>          ..
               0 File(s)              0 bytes

     Total Files Listed:
              17 File(s)         26,400 bytes
              20 Dir(s)  429,329,657,856 bytes free

D:\aeonSolutions\WebCoder-Multi-Platform-Framework>
AkashRajpurohit commented 3 months ago

i did a git-sync again and now the error is on another repository image

Can you run this with --log-level=debug so we can get some additional logs about the error output.

github-actions[bot] commented 3 weeks ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

BuyMyMojo commented 3 weeks ago

Running this on an Arch Linux system system and I got the same issue.

output with --log-level=debug:

{"level":"debug","time":"2024-11-06T21:04:53.554+1100","message":"Output: Cloning into bare repository '/run/media/buymymojo/ExternalLinuxHDD/git-sync/BuyMyMojo/RustDrop/RustDrop.git'...\nerror: RPC failed; curl 92 HTTP/2 stream 7 was not closed cleanly: CANCEL (err 8)\nerror: 13482 bytes of body are still expected\nfetch-pack: unexpected disconnect while reading sideband packet\nfatal: early EOF\nfatal: fetch-pack: invalid index-pack output\n\n"}
{"level":"fatal","time":"2024-11-06T21:04:53.554+1100","message":"Error cloning repo BuyMyMojo/RustDrop: exit status 128\n","stacktrace":"github.com/AkashRajpurohit/git-sync/pkg/logger.Fatalf\n\t/home/buymymojo/go/pkg/mod/github.com/!akash!rajpurohit/git-sync@v0.11.1/pkg/logger/logger.go:96\ngithub.com/AkashRajpurohit/git-sync/pkg/sync.CloneOrUpdateRepo\n\t/home/buymymojo/go/pkg/mod/github.com/!akash!rajpurohit/git-sync@v0.11.1/pkg/sync/sync.go:70\ngithub.com/AkashRajpurohit/git-sync/pkg/github.GitHubClient.Sync.func1\n\t/home/buymymojo/go/pkg/mod/github.com/!akash!rajpurohit/git-sync@v0.11.1/pkg/github/github.go:51"}

again it's on line 51 of github.go, I can re-run the command and it seems to happen on a different repo with each attempt.

seems like it could be some kind of rate limit cutting the connection on GitHub's end?

My config:

backup_dir: /run/media/buymymojo/ExternalLinuxHDD/git-sync/
clone_type: mirror
cron: ""
exclude_orgs: ["NVIDIAGameWorks", "gadsu"]
exclude_repos: []
include_forks: true
include_orgs: []
include_repos: []
include_wiki: true
platform: github
server:
    domain: github.com
    protocol: https
token: "No~" # Expires Dec 6 2024
username: "BuyMyMojo"
workspace: ""
AkashRajpurohit commented 3 weeks ago

It can possibly be rate limiting, can you try creating a different token or try after sometime when the rate limit would have been reset.

BuyMyMojo commented 3 weeks ago

It can possibly be rate limiting, can you try creating a different token or try after sometime when the rate limit would have been reset.

I have tried two tokens and attempted at different times so I think the stopping at different repos is the rate limit being tripped differently. maybe a small delay between repo pulls could fix this