[4.0] Update over stream fails (with curl disabled)

[Harmageddon]

Steps to reproduce the issue

1. Disable cURL support on your system. You might either actually disable it in your PHP configuration, or, more simple, change the function CurlTransport::isSupported (libraries/src/Http/Transport/CurlTransport.php, line 296) to:

   public static function isSupported()
   {
       return false; // Delete this after testing.
       return \function_exists('curl_version') && curl_version();
   }

2. Go to System - Joomla! Update.
3. If you are running a 4.0-beta version, you might need to change the update channel to testing and stability to beta.
4. Check for updates.
5. If you don't have the most recent version, try to update. If you already have the most recent version, try the "Reinstall core files" functionality as it uses the same procedures.

Expected result

The update should work using the stream protocol.

Actual result

Update over the stream protocol fails.

An error has occurred.

0 "" is not valid header name 

Call stack

1   ()  JROOT/libraries/vendor/laminas/laminas-diactoros/src/HeaderSecurity.php:169
2   Laminas\Diactoros\HeaderSecurity::assertValidName()     JROOT/libraries/vendor/laminas/laminas-diactoros/src/MessageTrait.php:414
3   Laminas\Diactoros\Response->assertHeader()  JROOT/libraries/vendor/laminas/laminas-diactoros/src/MessageTrait.php:341
4   Laminas\Diactoros\Response->setHeaders()    JROOT/libraries/vendor/laminas/laminas-diactoros/src/Response.php:135
5   Laminas\Diactoros\Response->__construct()   JROOT/libraries/src/Http/Transport/StreamTransport.php:232
6   Joomla\CMS\Http\Transport\StreamTransport->getResponse()    JROOT/libraries/src/Http/Transport/StreamTransport.php:200
7   Joomla\CMS\Http\Transport\StreamTransport->request()    JROOT/libraries/vendor/joomla/http/src/Http.php:314
8   Joomla\Http\Http->makeTransportRequest()    JROOT/libraries/vendor/joomla/http/src/Http.php:152
9   Joomla\Http\Http->get()     JROOT/administrator/components/com_joomlaupdate/src/Model/UpdateModel.php:452
10  Joomla\Component\Joomlaupdate\Administrator\Model\UpdateModel->downloadPackage()    JROOT/administrator/components/com_joomlaupdate/src/Model/UpdateModel.php:352
11  Joomla\Component\Joomlaupdate\Administrator\Model\UpdateModel->download()   JROOT/administrator/components/com_joomlaupdate/src/Controller/UpdateController.php:59
12  Joomla\Component\Joomlaupdate\Administrator\Controller\UpdateController->download()     JROOT/libraries/src/MVC/Controller/BaseController.php:729 

System information (as much as possible)

PHP Version 7.3.22 Web Server nginx/1.18.0 WebServer to PHP Interface fpm-fcgi

cURL disabled

Additional comments

The headers retrieved for the update package are as follows:

‌HTTP/1.1 302 Found
date: Fri, 06 Nov 2020 13:56:36 GMT
content-type: text/html; charset=utf-8
server: GitHub.com
status: 302 Found
vary: X-PJAX, Accept-Encoding, Accept, X-Requested-With, Accept-Encoding
location: https://github-production-release-asset-2e65be.s3.amazonaws.com/[...]&response-content-disposition=attachment%3B%20filename%3DJoomla_4.0.0-beta5-Beta-Update_Package.zip&response-content-type=application%2Foctet-stream
cache-control: no-cache
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: no-referrer-when-downgrade
expect-ct: max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"
content-security-policy: default-src 'none'; [...]
Set-Cookie: [...]; Path=/; HttpOnly; Secure; SameSite=Lax
Set-Cookie: [...]; Path=/; Domain=github.com; Expires=Sat, 06 Nov 2021 12:56:38 GMT; Secure; SameSite=Lax
Set-Cookie: logged_in=no; Path=/; Domain=github.com; Expires=Sat, 06 Nov 2021 12:56:38 GMT; HttpOnly; Secure; SameSite=Lax
Content-Length: 664
Connection: close
X-GitHub-Request-Id: 27DA:CE85:3DB0F4:5492C9:5FA55616
HTTP/1.1 200 OK
x-amz-id-2: fpMtb93sOduua6wEWxxdvdy7UkF5srpksVXnQhCeyI1dPc/E0WuzQ5Da5zholZ4+l7BweU9cVHE=
x-amz-request-id: 0C343098F891221A
Date: Fri, 06 Nov 2020 13:56:39 GMT
Last-Modified: Tue, 27 Oct 2020 20:03:19 GMT
ETag: "b308a29294f1bbeff68fdaf272a22e49"
Content-Disposition: attachment; filename=Joomla_4.0.0-beta5-Beta-Update_Package.zip
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 24391790
Server: AmazonS3
Connection: close

As you can see, we have two blocks there: One initiated with a 302 redirect status code, and one initiated with a 200 found status code.

In CurlTransport::getResponse, we have this section: https://github.com/joomla/joomla-cms/blob/e586555c9920b61b139ac9a113e48f5dc22ac2b4/libraries/src/Http/Transport/CurlTransport.php#L235-L272

There, the case of having one or more redirects and thus, several header blocks, is covered. Can we just replicate this for StreamTransport? As I'm not an expert in this field, I'm hesitant to just creating a PR without knowing possible sideeffects.

[astridx]

Please see https://github.com/joomla/joomla-cms/issues/23235

[Harmageddon]

Please see #23235

@astridx Thank you, but I think that's a different issue. #23235 is about not properly detecting if curl is disabled in some cases. The issue here is that the alternative to curl doesn't work for core updates (after it has been properly detected that curl is not available). Other things that don't use redirects, like querying the update information, work fine.

[astridx]

Yes you are right: "different issue". I mainly linked #23235 because it is also about the question of whether we want to support "deactivated curl systems" at all.

[Harmageddon]

Ah, I see. We currently do support other protocols than cURL and I think we should continue doing so.