search

sylabs / scs-library-client

Go client for the Singularity Container Services (SCS) Container Library Service
Other
4 stars 9 forks source link

Improve detection of HTTP "Range" requests in ConcurrentDownloadImage #141

Open tri-adam opened 2 years ago

tri-adam commented 2 years ago

The client supports parallel downloads using HTTP Range Requests. Currently, the client assumes ranged requests are supported if and only if the server performs a redirect via a http.StatusSeeOther. Otherwise, c.singleStreamDownload is utilized:

https://github.com/sylabs/scs-library-client/blob/2e30d58f8a9212b4a35946eb62d42f1e84140f0c/client/pull.go#L273-L278

This negatively impacts download performance against servers that support ranged requests, but do not redirect via http.StatusSeeOther, and is the root cause of #129.

The current logic should be replaced with logic that detects HTTP range support in the standard fashion (ref).

EmmEff commented 1 year ago

This currently affects (only) Singularity Enterprise 2.0.x, which does not redirect on the /v1/imagefile endpoint.