search

swiftlang / swift-package-manager

The Package Manager for the Swift Programming Language
Apache License 2.0
9.74k stars 1.34k forks source link

Package collections served with a chunked response cannot be added #5553

Open JosephDuffy opened 2 years ago

JosephDuffy commented 2 years ago

Expected behavior

Servers responding with a Transfer-Encoding: chunked should be supported.

Actual behavior

"Missing Content-Length header" error is displayed.

Steps to reproduce

  1. Run swift package-collection add with a Swift Package Collection that supports chunked encoding

If possible, minimal yet complete reproducer code (or URL to code)

swift package-collection add <url that supports chunking>

For example, hosting with Next.js will support chunked responses for static files by default.

Swift Package Manager version/commit hash

Current main (c0d5daa8fb43d87a6043a79c3cd132987eb40623), whatever is shipped with Xcode 13.2.1

Swift & OS version (output of swift --version && uname -a)

swift-driver version: 1.26.21 Apple Swift version 5.5.2 (swiftlang-1300.0.47.5 clang-1300.0.29.30)
Target: arm64-apple-macosx12.0
Darwin M1-MacBook-Pro.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64

More Info

The Accept-Encoding header is being set to deflate, identity, gzip;q=0: https://github.com/apple/swift-package-manager/blob/main/Sources/PackageCollections/Providers/JSONPackageCollectionProvider.swift#L305. When gzip or deflate are used the response may be chunked, which causes the Content-Length header to not be included.

neonichu commented 2 years ago

cc @yim-lee

tomerd commented 2 years ago

we can probably relax the requirement on content-length header when detecting streamed content as we have a secondary size protection for streamed payloads