SwiftPM can't properly download a binary dependency

[nkbelov]

Description

When adding https://bitbucket.org/usercentricscode/usercentrics-spm-sdk (a dependency that itself declares a zipped binary target), SwiftPM is unable to download the archive completely, reporting

Fetching https://bitbucket.org/usercentricscode/usercentrics-spm-sdk from cache
Fetched https://bitbucket.org/usercentricscode/usercentrics-spm-sdk (0.66s)
Computing version for https://bitbucket.org/usercentricscode/usercentrics-spm-sdk
Computed https://bitbucket.org/usercentricscode/usercentrics-spm-sdk at 2.8.1 (0.87s)
Creating working copy for https://bitbucket.org/usercentricscode/usercentrics-spm-sdk
Working copy of https://bitbucket.org/usercentricscode/usercentrics-spm-sdk resolved at 2.8.1
Downloading binary artifact https://bitbucket.org/usercentricscode/usercentrics-spm-sdk/downloads/Usercentrics-2.8.1.xcframework.zip
error: invalid archive returned from 'https://bitbucket.org/usercentricscode/usercentrics-spm-sdk/downloads/Usercentrics-2.8.1.xcframework.zip' which is required by binary target 'Usercentrics'

When investigating .build/artifacts/usercentrics-spm-sdk/Usercentrics, the Usercentrics-2.8.1.xcframework.zip file is not loaded completely (or correctly): wc reports only 6720 bytes, whereas the full archive is 12.4 MB.

unzip doesn't work on it as a result with:

End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.

Downloading the archive by opening the link in a browser or by curl -L -O-ing the link loads the full archive, and unzip-ing it succeeds.

Expected behavior

The package loads, the archive unzips and the primary target compiles with no errors.

Actual behavior

See description.

Steps to reproduce

1. Create an empty Swift executable and add

       .package(url: "https://bitbucket.org/usercentricscode/usercentrics-spm-sdk", from: "2.0.0")

   to dependencies.

2. Run swift build.

Swift Package Manager version/commit hash

Swift Package Manager - Swift 5.8.0

Swift & OS version (output of swift --version ; uname -a)

swift-driver version: 1.75.2 Apple Swift version 5.8 (swiftlang-5.8.0.124.2 clang-1403.0.22.11.100) Target: arm64-apple-macosx13.0 Darwin *** 22.5.0 Darwin Kernel Version 22.5.0: Mon Apr 24 20:53:44 PDT 2023; root:xnu-8796.121.2~5/RELEASE_ARM64_T8103 arm64

[maheshbijapur]

Even I am seeing same error, any update on this?

[Kyle-Ye]

Can't reproduce this issue.

Xcode Version 15.0 beta 4 (15A5195m)

swift-driver version: 1.85 Apple Swift version 5.9 (swiftlang-5.9.0.120.7 clang-1500.0.34.3)
Target: arm64-apple-macosx14.0

Maybe there is something wrong with your network connection or the cache.

You can remove all the cache by the following commands if you are on macOS.

rm -rf $HOME/Library/Caches/org.swift.swiftpm/
rm -rf $HOME/Library/org.swift.swiftpm
rm -rf $HOME/Library/Developer/Xcode/DerivedData

[ekin-celik-deltatre]

I had the same issue. I solved it by removing my bitbucket username/password from keychain.

[SolomiyaN]

I have the same issue. @ekin-celik-deltatre what do you mean — have you removed the username/password altogether? Or did you add them back later?

[ekin-celik-deltatre]

hi @SolomiyaN , I opened Keychain Access and searched for bitbucket on Passwords. Removed it and run resolve packages again. It worked for me. Then I added it back.

[nickx3389]

@nkbelov I also have the same issue. Did you manage to get it done? @ekin-celik-deltatre Your solution did not work unfortunately...

[sfunke]

I had the same issue, turned out I had a Bitbucket App password in my macOS keychain (named bitbucket.org), probably some leftover from whatever ... - I deleted it, and voila, it worked again. So far no side effects.

[SpessotoVitor97]

I'm facing the same issue, but instead of bitbucket I'm using GitHub.

Neither @SolomiyaN nor @Kyle-Ye solutions have worked for me.

[SolomiyaN]

Hey @SpessotoVitor97 I have ended up downloading a certain version of UserCentric and adding it locally to the project.

[Kyle-Ye]

I'm facing the same issue, but instead of bitbucket I'm using GitHub.

Neither @SolomiyaN nor @Kyle-Ye solutions have worked for me.

Could you provide the binary package url or your Package.swift and your OS/Xcode version here? So that I can help try to reproduce the bug potentially on my machine.

cc @SpessotoVitor97 @SolomiyaN

[Kristijan22]

I also just faced the same issue on Xcode 15.3. but with a different dependency. Never encountered it on Xcode 15.2. or before.

Running

rm -rf $HOME/Library/Caches/org.swift.swiftpm/
rm -rf $HOME/Library/org.swift.swiftpm

and reloading the packages worked for me.

[kevinkleine]

Potential same issue here while trying to build a package on Linux with this dependency:

let package = Package(
    name: "godotest",
    dependencies: [
        .package(url: "https://github.com/migueldeicaza/SwiftGodotKit", revision: "a7f90ce4fa3e20fa95b57f1d95704445e5844d3d")
    ],
    targets: [
        .executableTarget(
            name: "godotest"),
    ]
)

swift build has the following output:

Fetching https://github.com/migueldeicaza/SwiftGodotKit
Fetched https://github.com/migueldeicaza/SwiftGodotKit (1.04s)
Fetching https://github.com/migueldeicaza/SwiftGodot
Fetched https://github.com/migueldeicaza/SwiftGodot (13.93s)
Fetching https://github.com/CoreOffice/XMLCoder
Fetched https://github.com/CoreOffice/XMLCoder (1.51s)
Computing version for https://github.com/CoreOffice/XMLCoder
Computed https://github.com/CoreOffice/XMLCoder at 0.17.1 (0.22s)
Creating working copy for https://github.com/CoreOffice/XMLCoder
Working copy of https://github.com/CoreOffice/XMLCoder resolved at 0.17.1
Creating working copy for https://github.com/migueldeicaza/SwiftGodot
Working copy of https://github.com/migueldeicaza/SwiftGodot resolved at 1359586f2332d500d30661158e062a0e9f97751c
Creating working copy for https://github.com/migueldeicaza/SwiftGodotKit
Working copy of https://github.com/migueldeicaza/SwiftGodotKit resolved at a7f90ce4fa3e20fa95b57f1d95704445e5844d3d
Downloading binary artifact https://github.com/migueldeicaza/SwiftGodotKit/releases/download/v1.0.1/libgodot.xcframework.zip
error: failed downloading 'https://github.com/migueldeicaza/SwiftGodotKit/releases/download/v1.0.1/libgodot.xcframework.zip' which is required by binary target 'libgodot': downloadError("Error Domain=NSURLErrorDomain Code=-1001 \"(null)\"")

Removing the contents of the .build dir within the package and the ~/.swiftpm/cache dir doesn't make a difference. When I download the file directly, it takes about 20 s. If someone can tell me how to manually add it to the package that would be a helpful workaround.

Swift version 5.9.2 OS Manjaro Linux 23.1.3

[MaxDesiatov]

Is this still reproducible for you with Swift 5.10 and/or latest development snapshots?

[kevinkleine]

Unfortunately yes:

~/p/k/swift/godotest > swift -v
Swift version 5.10 (swift-5.10-RELEASE)
Target: x86_64-unknown-linux-gnu
/usr/lib/swift/bin/swift-help intro

Welcome to Swift!

Subcommands:

  swift build      Build Swift packages
  swift package    Create and work on packages
  swift run        Run a program from a package
  swift test       Run package tests
  swift repl       Experiment with Swift code interactively

  Use `swift --version` for Swift version information.

  Use `swift --help` for descriptions of available options and flags.

  Use `swift help <subcommand>` for more information about a subcommand.

~/p/k/swift/godotest > swift package reset
~/p/k/swift/godotest > swift build
Fetching https://github.com/migueldeicaza/SwiftGodotKit from cache
Fetched https://github.com/migueldeicaza/SwiftGodotKit from cache (0.34s)
Fetching https://github.com/migueldeicaza/SwiftGodot from cache
Fetched https://github.com/migueldeicaza/SwiftGodot from cache (0.41s)
Fetching https://github.com/CoreOffice/XMLCoder from cache
Fetched https://github.com/CoreOffice/XMLCoder from cache (0.36s)
Computing version for https://github.com/CoreOffice/XMLCoder
Computed https://github.com/CoreOffice/XMLCoder at 0.17.1 (0.23s)
Creating working copy for https://github.com/migueldeicaza/SwiftGodotKit
Working copy of https://github.com/migueldeicaza/SwiftGodotKit resolved at a7f90ce4fa3e20fa95b57f1d95704445e5844d3d
Creating working copy for https://github.com/CoreOffice/XMLCoder
Working copy of https://github.com/CoreOffice/XMLCoder resolved at 0.17.1
Creating working copy for https://github.com/migueldeicaza/SwiftGodot
Working copy of https://github.com/migueldeicaza/SwiftGodot resolved at 1359586f2332d500d30661158e062a0e9f97751c
Downloading binary artifact https://github.com/migueldeicaza/SwiftGodotKit/releases/download/v1.0.1/libgodot.xcframework.zip
error: failed downloading 'https://github.com/migueldeicaza/SwiftGodotKit/releases/download/v1.0.1/libgodot.xcframework.zip' which is required by binary target 'libgodot': downloadError("Error Domain=NSURLErrorDomain Code=-1001 \"(null)\"")

[ferologics]

ran into the same issue just now, the downloaded archive is just 9 bytes but is supposed to be 65.9 megabytes.

invalid archive returned from 'https://github.com/hypertrack/sdk-ios/releases/download/5.5.0/HyperTrack.xcframework.zip' which is required by binary target 'HyperTrack'

[ferologics]

and yes, nuking spm and xcode cache fixes this

[thedavidharris]

Also seeing this, solely with Xcode 15.3/Swift 10. Resolving in Xcode 15.2 works fine, the xcframework can be reused in 15.3, but if you clear cache and try to resolve in 15.3 it reliably gives this error.

Clearing all caches did not solve the issue.

[morale3232]

I'm getting this same error after following the official tutorial from Kotlin regarding exporting Multiplatform libraries to XCFrameworks (https://kotlinlang.org/docs/native-spm.html#create-the-xcframework-and-the-swift-package-manifest). I'm using Xcode 15.4 and Swift 5.10. I've cleared caches to no avail. I don't have Bitbucket credentials in my Keychain. I didn't try removing all my Github credentials from Keychain, but I suppose I can try that if we think it may do the trick.

Any ideas to resolve this error besides wiping out Bitbucket credentials?

[andreashanft]

Hi, I'm seeing this issue as well randomly for our binary dependencies ("invalid archive returned from ...") served from an internal Artifactory instance. Looking at the downloaded data in ~/Library/Caches/org.swift.swiftpm/artifacts shows the file is just a few bytes and the contents are plaintext/HTML, looks like auth errors are not handled properly when downloading binary dependencies.:

<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
</body>
</html>

Simply deleting the corresponding cached (and corrupt) file from the artifacts folder is enough for me to fix the issue. No need to nuke/reset the whole SPM cache.

[rodrigoelp]

Hello!

Just following on this. I have found that one of reasons for failure here stems off how some credentials are used to resolve the binary targets, but more importantly how the error is communicated to the developer.

In ZipArchiver (or any archiver to be honest), the process is launched and upon failure, it is communicated through the result, but never inspected (and the problem is the zip file contains the error).

One improvement we can make is to change https://github.com/swiftlang/swift-package-manager/blob/f55fbef52bbdc79d5c790ef3a020a40ac0952bd3/Sources/Workspace/Workspace%2BBinaryArtifacts.swift#L306 to check if the content of the file aren't an archive (maybe process execute file with the archivePath) and print the output of the file in the case it isn't an archive for the developer to get the content of the error message.

OR we could change the way the ZipArchiver (https://github.com/swiftlang/swift-package-manager/blob/f55fbef52bbdc79d5c790ef3a020a40ac0952bd3/Sources/Basics/Archiver/ZipArchiver.swift#L69) works to do a similar verification to allow developers to understand what's going wrong instead of the invalid archive returned.

I have noticed that this can fail if you have services like aws configured in your keychain, as well as artifacts in aws. For some reason I haven't found yet, spm tries to authenticate with the credentials in keychain, which is a kind of fair assumption I guess.