search

eclipse-jgit / jgit

JGit, the Java implementation of git
https://www.eclipse.org/jgit/
Other
117 stars 33 forks source link

Pushing LFS objects to GitHub fails #11

Open HannesWell opened 8 months ago

HannesWell commented 8 months ago

Version

6.9.0

Operating System

Windows

Bug description

Trying to push a commit to GitHub that contains a new Large File Storage (LFS) object fails with a message that says that the referenced LFS object is missing grafik

But the object exists in the local LFS cache and I also debugged the code and verified that LfsPrePushHook.uploadFile() is called and succeeds for the file mentioned in the error message. Switching between the Apache Http and the JDK's http client makes no difference.

grafik

Actual behavior

Pushing failes with the described message.

Expected behavior

Push succeeds and the LFS objects are properly pushed along with the pointer file.

Relevant log output

Repository git@github.com:HannesWell/my.git.lfs.tests.git

pre-receive hook declined
  error: GH008: Your push referenced at least 1 unknown Git LFS object:
    cdb81c2552ec215adc17feb7665f9b7532beec6735816a040798207dc0e71994
Try to push them with 'git lfs push --all'.

Other information

The repositories .git/config:

[core]
    repositoryformatversion = 0
    filemode = false
    logallrefupdates = true
[remote "origin"]
    url = git@github.com:HannesWell/my.git.lfs.tests.git
    fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
    merge = refs/heads/main
    rebase = false
    remote = origin
[lfs]
    repositoryformatversion = 0

My global ~/.gitconfig

[http]
    sslVerify = false
[push]
    default = simple
[core]
    autocrlf = true
    safecrlf = true
[filter "lfs"]
    useJGitBuiltin = true
    required = true
    smudge = git-lfs smudge -- %f
    process = git-lfs filter-process
    clean = git-lfs clean -- %f
HannesWell commented 8 months ago

If I try to push via git CLI (which ships with git-lfs by default on windows) I get the following error

$ git push
To github.com:HannesWell/my.git.lfs.tests.git
 ! [rejected]        main -> main (fetch first)
error: failed to push some refs to 'github.com:HannesWell/my.git.lfs.tests.git'
hint: Updates were rejected because the remote contains work that you do not
hint: have locally. This is usually caused by another repository pushing to
hint: the same ref. If you want to integrate the remote changes, use
hint: 'git pull' before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

But if I fetch I see no other commits or remote reference updates.

So it looks like the push is not fully rejected but somehow incomplete.

HannesWell commented 8 months ago

If I try to push via git CLI (which ships with git-lfs by default on windows) I get the following error

From some more experiments I've got the impression that this is not caused by an imcomplet push through JGit, but also seem to happen when one pushes to a fork of a repository. At least in that case pushes of new LFS-objects to new branches lead to the same error with JGit and CLI git. Adding a .lfsconfig like the following makes CLI git work (in this case to push to my SWT clone):

[lfs]
    url = https://lfs.github.com/eclipse-platform/eclipse.platform.swt

But then JGit fails with the following error if I try to push some LFS-objects:

eclipse.buildId=4.31.0.I20231230-1800
java.version=17.0.7
java.vendor=GraalVM Community
BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=de_DE
Command-line arguments:  -os win32 -ws win32 -arch x86_64

org.eclipse.egit.core
Error
Wed Jan 03 00:52:55 CET 2024
An exception occurred during push on URI git@github.com:HannesWell/eclipse.platform.swt.git: When trying to open a connection to https://lfs.github.com/eclipse-platform/eclipse.platform.swt/objects/batch the server responded with an error code. rc=401

org.eclipse.jgit.errors.TransportException: When trying to open a connection to https://lfs.github.com/eclipse-platform/eclipse.platform.swt/objects/batch the server responded with an error code. rc=401
    at org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:174)
    at org.eclipse.jgit.transport.Transport.push(Transport.java:1557)
    at org.eclipse.egit.core.op.PushOperation.run(PushOperation.java:269)
    at org.eclipse.egit.ui.internal.push.PushJob.performJob(PushJob.java:86)
    at org.eclipse.egit.ui.internal.jobs.RepositoryJob.run(RepositoryJob.java:59)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: java.io.IOException: When trying to open a connection to https://lfs.github.com/eclipse-platform/eclipse.platform.swt/objects/batch the server responded with an error code. rc=401
    at org.eclipse.jgit.lfs.LfsPrePushHook.requestBatchUpload(LfsPrePushHook.java:200)
    at org.eclipse.jgit.lfs.LfsPrePushHook.call(LfsPrePushHook.java:105)
    at org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:172)
    ... 5 more

I also debugged this from an J/EGit workspace, but I'm puzzled why the lfs.url available anywhere else than LfsConnectionFactory.getLfsUrl()? Could it be that looking up the LF storage using LfsConnectionFactory.getSshAuthentication() respectively git-lfs-authenticate somehow creates a session or alike that is used to also authenticate the subsequent LFS-object upload?

dispeide commented 3 weeks ago

I have the same issue with external lfs storage using ssh connection:

In our environment we try to use external lfs storage via ssh connection. This works well in command line but not within eclipse / jgit. There we get the error message unknown protocol: ssh

configuration eample: .lfsconfig

[lfs]
  url = ssh://git@<jfrog-artifactory-url>:<port>/artifactory/lfs-reopsitory

ussed environment: git: Bitbucket lfs-storage: jfrog artifactory