Open swift-ci opened 6 years ago
@swift-ci create
Has there been any update on this issue? There's a similar discussion at https://forums.swift.org/t/swiftpm-with-git-lfs/42396.
I am running into this issue as well with https://github.com/foxglove/mcap. This dependency fails during checkout: .package(url: "https://github.com/foxglove/mcap", branch: "main"),
Creating working copy for https://github.com/foxglove/mcap
error: Couldn’t check out revision ‘2dad37cad3c7875842b44f3fe927e2dcfdecf7d4’:
Downloading docs/img/0398582-hero.png (669 KB)
Error downloading object: docs/img/0398582-hero.png (266f9d3): Smudge error: Error downloading docs/img/0398582-hero.png (266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5): error transferring "266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5": [0] remote missing object 266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5
Errors logged to /Users/jacob/Desktop/testpackage/.build/checkouts/mcap/.git/lfs/logs/20220923T112418.927225.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: docs/img/0398582-hero.png: smudge filter lfs failed
A simple git clone
succeeds.
Is there any progress on it or someone has a solution?
We're talking about an Xcode specific issue here, correct? I tried reproducing with https://github.com/foxglove/mcap and swift build
and it seems to work fine for me.
Nope, it's not an Xcode specific issue. swift build
within the repository works fine when you manually clone it. The problem is with swiftpm's automatic cloning.
Try creating an empty directory and add this Package.swift:
// swift-tools-version:5.5
import PackageDescription
let package = Package(
name: "demo",
products: [],
dependencies: [
.package(url: "https://github.com/foxglove/mcap", branch: "main"),
],
targets: []
)
Then run swift build
:
$ swift build
Fetching https://github.com/foxglove/mcap from cache
Fetched https://github.com/foxglove/mcap (2.03s)
Fetching https://github.com/apple/swift-collections from cache
Fetched https://github.com/apple/swift-collections (0.76s)
Fetching https://github.com/apple/swift-algorithms from cache
Fetched https://github.com/apple/swift-algorithms (0.33s)
Computing version for https://github.com/apple/swift-algorithms
Computed https://github.com/apple/swift-algorithms at 1.0.0 (0.32s)
Fetching https://github.com/apple/swift-numerics from cache
Fetched https://github.com/apple/swift-numerics (0.60s)
Computing version for https://github.com/apple/swift-numerics
Computed https://github.com/apple/swift-numerics at 1.0.2 (0.33s)
Creating working copy for https://github.com/apple/swift-algorithms
Working copy of https://github.com/apple/swift-algorithms resolved at 1.0.0
Creating working copy for https://github.com/apple/swift-collections
Working copy of https://github.com/apple/swift-collections resolved at 418378107c87a4b312e29a51f773ce0e4e12e199
Creating working copy for https://github.com/apple/swift-numerics
Working copy of https://github.com/apple/swift-numerics resolved at 1.0.2
Creating working copy for https://github.com/foxglove/mcap
error: Couldn’t check out revision ‘14fa18afb83e504ae03397fd4e9292a6c83a7e80’:
Downloading docs/img/0398582-hero.png (669 KB)
Error downloading object: docs/img/0398582-hero.png (266f9d3): Smudge error: Error downloading docs/img/0398582-hero.png (266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5): error transferring "266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5": [0] remote missing object 266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5
Errors logged to /Users/jacob/Desktop/swifttest/.build/checkouts/mcap/.git/lfs/logs/20221006T101212.79192.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: docs/img/0398582-hero.png: smudge filter lfs failed
Interesting, this does not happen for me. I ended up with the same commit as you, but nothing in .git/lfs
:
❯ git show --oneline -s
14fa18afb (HEAD, origin/main, origin/HEAD, main) fix link in README (#636)
/tmp/foo/.build/checkouts/mcap heads/main
❯ ls -ld .git/lfs
".git/lfs": No such file or directory (os error 2)
I'm not really familiar with git-lfs
, is there any global config required for it?
I'm not really familiar with
git-lfs
, is there any global config required for it?
Yes, normally users will run git lfs install
once, as described here: https://git-lfs.github.com
It's a very common thing to have installed, since many projects use Git LFS.
What do you see when you look at /tmp/foo/.build/checkouts/mcap/docs/img/0398582-hero.png
? I'm guessing you have only the LFS pointer file, which is what you'd get if you don't have LFS installed.
Also note that some folks on Discourse have dug deeper into why this happens: https://forums.swift.org/t/swiftpm-with-git-lfs/42396/13
Thanks, this helps!
Thanks for looking into this! 🙌 Just wondering if there’s been any update so far on a possible resolution?
I tried adding a .lfsconfig file to the repo, as recommended in the Discourse thread, and this seems to have worked around the issue for swift build
(like the small example I gave you), but it still doesn't work when adding the dependency in Xcode...
Example project that fails to build in Xcode on my machine: TestPackage.zip
Based on this thread and the forum discussion, I came to the same blocking errors.
Linking git-lfs did work in the sense that Xcode used it but reported a different error downloading various files that were checked in with lfs. The error included "Smudge error: Error downloading..."
> cd /Applications/Xcode-14.1.0.app/Contents/Developer
> ln -s /opt/homebrew/bin/git-lfs git-lfs
Adding a .lfsconfig (with an updated path for my repo) did not help with me when running swift build
or an Xcode package resolution. I did have some GitHub authentication errors until I removed the commit that added the file.
The only work around I have is to manually clone the repo and include it as a local package—which of course negates all advantages of using a dependency manager. 😫
I did file FB11803523 for Xcode.
Not sure if it helps you, but I believe the .lfsconfig file has to be present on the repo's default branch. When SPM clones the package it starts with the default branch and then checks out whichever branch/commit you requested.
This did ultimately end up working well enough for me, I was able to use xcodebuild
in CI (but not the Xcode app): https://github.com/foxglove/MCAPSpotlightImporter/actions/runs/3380647041/jobs/5613664270
However, the .lfsconfig file is causing some other issues for developers working with the repo... 😓
Is there any update to this?
+1
+1
One option for this I wrote up here (which only applies if the LFS files are something optional for your package, like test results): https://stackoverflow.com/questions/67457594/swift-package-manager-does-not-work-with-git-lfs/77336212#77336212
The workaround from @briancordanyoung seem to work, but the Xcode path has changed
cd /Applications/Xcode.app/Contents/Developer/usr/bin
ln -s /opt/homebrew/bin/git-lfs git-lfs
Seems the workaround conflict with SIP, it's weird to include XCode folder in SIP.
Are you sure it is SIP @yangkx1024? Wonder why the symlinking was allowed for me, I have not disabled SIP
csrutil status
System Integrity Protection status: enabled.
I disabled the SIP and the ln
command success, but now I stuck with the git-lfs
issue, same as this post: https://forums.swift.org/t/swiftpm-with-git-lfs/42396/13
@neonichu if you learned anything about the root cause of this issue or possible workaround, I’m sure many of us would be interested to hear it, thanks :)
I am using the following workaround:
Switch to using your system Git.
Run the following command from Terminal, from the directory where your .xcodeproj is located, as appropriate for your project. It will resolve the package through your system Git. Your system Git needs to already have Git LFS configured.
xcodebuild -scmProvider system -resolvePackageDependencies
Note: You can also add
-project
,-workspace
, and-scheme
arguments as appropriate to target your development setup. Refer to the xcodebuild (Command Line Tools) FAQ and-help
section.
Can we get an update on this please? It's been open for 6 years but seems like a critical barrier preventing 100% adoption of Swift PM.
@iosdevben I would love to have this fixed. But I would not let it prevent you from adopting SPM. I've been using this work around all this time and the benefits of SPM have made it completely worth it.
Additional Detail from JIRA
| | | |------------------|-----------------| |Votes | 2 | |Component/s | Package Manager | |Labels | Bug | |Assignee | None | |Priority | Medium | md5: 2218c2c1d09b919e983b204e24d25058Issue Description:
The error is in the format of: (XXXXX being the name of LFS-tracked file)
This happens during the "cloneCheckout" phase i.e. invoking "git clone --shared". If I disable the LFS filter by editing the global .gitconfig, the project builds fine.