search

eclipse-jgit / jgit

JGit, the Java implementation of git
https://www.eclipse.org/jgit/
Other
92 stars 31 forks source link

Fetch with protocol V2 fail if wanted ref has been updated during fetch #48

Open thomasdraebing opened 3 months ago

thomasdraebing commented 3 months ago

Version

6.9.0

Operating System

Linux/Unix, MacOS

Bug description

When a large repository with a lot of refs and activity is being, it can happen that the following error is being thrown:

Caused by: org.eclipse.jgit.transport.WantNotValidException: want 9784ba017f833fc41c1ad4344a75105ee69d639e not valid
    at org.eclipse.jgit.transport.UploadPack$AdvertisedRequestValidator.checkWants(UploadPack.java:2002)
    at org.eclipse.jgit.transport.UploadPack.parseWants(UploadPack.java:1953)
    at org.eclipse.jgit.transport.UploadPack.processHaveLines(UploadPack.java:1827)
    at org.eclipse.jgit.transport.UploadPack.fetchV2(UploadPack.java:1275)
    at org.eclipse.jgit.transport.UploadPack.serveOneCommandV2(UploadPack.java:1389)
    at org.eclipse.jgit.transport.UploadPack.serviceV2(UploadPack.java:1443)
    at org.eclipse.jgit.transport.UploadPack.uploadWithExceptionPropagation(UploadPack.java:886)
    at org.eclipse.jgit.transport.UploadPack.upload(UploadPack.java:796)
    ... 12 more

Initial research showed that this happens, if a wanted ref is being updated during the fetch. Looking at the code it seems that this change has to happen while the fetch spends time in UploadPack.fetchV2(), between the computation of wantedIds and advertised IDs, which happens later. In that case both Sets differ from each other and validation fails.

This was observed in Gerrit for fetches in large repositories with wide refSpecs or for mirror clones.

Actual behavior

Caused by: org.eclipse.jgit.transport.WantNotValidException: want 9784ba017f833fc41c1ad4344a75105ee69d639e not valid
    at org.eclipse.jgit.transport.UploadPack$AdvertisedRequestValidator.checkWants(UploadPack.java:2002)
    at org.eclipse.jgit.transport.UploadPack.parseWants(UploadPack.java:1953)
    at org.eclipse.jgit.transport.UploadPack.processHaveLines(UploadPack.java:1827)
    at org.eclipse.jgit.transport.UploadPack.fetchV2(UploadPack.java:1275)
    at org.eclipse.jgit.transport.UploadPack.serveOneCommandV2(UploadPack.java:1389)
    at org.eclipse.jgit.transport.UploadPack.serviceV2(UploadPack.java:1443)
    at org.eclipse.jgit.transport.UploadPack.uploadWithExceptionPropagation(UploadPack.java:886)
    at org.eclipse.jgit.transport.UploadPack.upload(UploadPack.java:796)
    ... 12 more

Expected behavior

The fetch or mirror clone works and all objects are being transferred.

Relevant log output

No response

Other information

No response

thomasdraebing commented 3 months ago

fyi @msohn