Prefetching packages: Failed to download from dist

[ihmels]

I have a problem installing dependencies in a Vagrant machine behind a corporate proxy.

The problem occurs when the packages are prefetched after Flex is downloaded:

Failed to download doctrine/lexer from dist: "https://api.github.com/repos/doctrine/lexer/zipball/83893c552fd2045dd78aef794c31e694c37c0b8c" appears broken, and returned an empty 200 response Now trying to download from source

If I disable Flex during installation with composer install --no-plugins everything is OK.

A similar problem also occurs with composer update (composer update --no-plugins is OK):

$ composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 86 installs, 0 updates, 0 removals
  - Installing symfony/flex (v1.2.1): Downloading (100%)
"https://repo.packagist.org/packages.json" does not contain valid JSON
Parse error on line 1:

^
Expected one of: 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
https://repo.packagist.org could not be fully loaded, package information was loaded from the local cache and may be out of date

In both cases Composer receives an empty response. Composer diagnose:

$ composer diagnose
Checking composer.json: OK
Checking platform settings: OK
Checking git settings: OK
Checking http connectivity to packagist: OK
Checking https connectivity to packagist: OK
Checking HTTP proxy: OK
Checking HTTP proxy support for request_fulluri: OK
Checking HTTPS proxy support for request_fulluri: OK
Checking github.com rate limit: OK
Checking disk free space: OK
Checking pubkeys: FAIL
Missing pubkey for tags verification
Missing pubkey for dev verification
Run composer self-update --update-keys to set them up
Checking composer version: OK
Composer version: 1.8.4
PHP version: 7.2.17
PHP binary path: /opt/remi/php72/root/usr/bin/php

[ihmels]

I found out something: The problem seems to occur since PHP 7.2.17. With PHP 7.2.16 it works fine.

[merigold]

PHP 7.3.4 problem exist. I try flex 1.2.1, 1.2.0, 1.1.8, the same

[loiclavoie]

Having the same issue. We also notice that the problem seems to diseapear if we disable tls.

composer config disable-tls false

Also, it seem's to only happen with dist on api.github.com. We have private one in our project and they don't have this issue.

[nicolas-grekas]

looks like a PHP bug can you reproduce when disabling the curl extension? or when enabling it?

[loiclavoie]

@nicolas-grekas curl extension cannot be removed on our server due to other features that requires it. But also, php-curl seem's to be part of php-commons now (with CentOS Remi-PHP7.2). So I cant' remove it.

[loiclavoie]

@nicolas-grekas If I don't install Flex, I don't get that error. Composer uses file_get_content to obtain a dist and flex is using curl. So it's seems to be indeed related to the usage of curl. Pass that point, I don't have more information

[merigold]

Whetn I try php 7.2.14 is ok, 7.2.17 and 7.3.4 cause this error. Workaraound composer config disable-tls true

[loiclavoie]

@merigold @ihmels I have make it work on our server for now, but it require downgrading php to version 7.2.16. Currently 7.2.17 contains an issue with CURL which cause the problem.

I've tested every bug declared on PHP.net and none of them seem's to be the cause.

You can downgrade if you are using Remi's package by doing

yum downgrade php-*

[ihmels]

@loiclavoie Unfortunately I can't downgrade PHP, because I just provisioned the VM.

@nicolas-grekas If I disable the cURL extension, it does work without problems.

[phoenix1337]

It looks like it has to do with the following fix which now returns an empty string instead of the buffer: https://github.com/php/php-src/commit/5025eb05bde83a0a51eb0668c45c240b366545bf

composer install with php 7.2.16 gives me the following errors: Retrying download: Unknown cipher in list: ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:E

If I execute curl --ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:E https://github.com/php/php-src/commit/5025eb05bde83a0a51eb0668c45c240b366545bf it gives me exactly the same error.

The ciphers are passed in https://github.com/symfony/flex/blob/7d488f0dc2ccd39d4f4603747614b0ed64d6f3c6/src/CurlDownloader.php#L33 It looks like it's working on 7.2.17 when this line is commented

I am not sure what causes this error exactly but this is what I found so far.

[nicolas-grekas]

Good catch @phoenix1337, would you mind sending a PR to remove this line? It's not needed anymore since default ciphers are usually good.

[remicollet]

php-curl seem's to be part of php-commons now (with CentOS Remi-PHP7.2). So I cant' remove it.

@loiclavoie you can disable it in 20-curl.ini

[remicollet]

to remove this line? It's not needed anymore since default ciphers are usually good.

@nicolas-grekas +1

Especially as changing default value from code is a terribly bad idea.

On OS where security is properly managed, this is event forbidden, (in packages distributed with the distribution), read https://www.redhat.com/en/blog/consistent-security-crypto-policies-red-hat-enterprise-linux-8

[phoenix1337]

I have created a pull request but it's failing because of coding standards on lines I did not change.

The problem with the constant exposing the ciphers is that it contains OpenSSL ciphers. On Centos the curl package is build with nss so they don't match. I don't know if this is a curl or php issue though.

[remicollet]

The problem with the constant exposing the ciphers is that it contains OpenSSL ciphers. On Centos the curl package is build with nss so they don't match. I don't know if this is a curl or php issue though.

I rather think that this constant is the option number, not its value....

curl_setopt($ch, CURLOPT_SSL_CIPHER_LIST, implode(':', $arrayCiphers));

[phoenix1337]

Correct. it's fetched from the parameters set on the stream. My bad.

[cpetit-sigma-fr]

I can confirm commenting out line 33 of vendor/symfony/flex/src/CurlDownloader.php solves the problem.

[nicolas-grekas]

Version v1.2.2 has been released a few hours ago with the fix, please upgrade.

[cpetit-sigma-fr]

Commenting out line 33 permitted me to upgrade Flex to 1.2.2, Thanks!

[melyouz]

This is till happening to us on Windows with v1.2.2, working behind an HTTP Proxy.

λ composer show | grep "symfony/flex"
symfony/flex                        v1.2.2   Composer plugin for Symfony

Output of composer diagnose:

λ composer diagnose
Checking composer.json: OK
Checking platform settings: OK
Checking git settings: OK
Checking http connectivity to packagist: OK
Checking https connectivity to packagist: OK
Checking HTTP proxy: OK
Checking HTTP proxy support for request_fulluri: OK
Checking HTTPS proxy support for request_fulluri: OK
Checking github.com rate limit: OK
Checking disk free space: OK
Checking pubkeys:
Tags Public Key Fingerprint: 57815BA2 7E54DC31 7ECC7CC5 573090D0  87719BA6 8F3BB723 4E5D42D0 84A14642
Dev Public Key Fingerprint: 4AC45767 E5EC2265 2F0C1167 CBBB8A2B  0C708369 153E328C AD90147D AFE50952
OK
Checking composer version: OK
Composer version: 1.8.5
PHP version: 7.3.4
PHP binary path: C:\php\php.exe

When composer is run with --no-plugins option, it works:

λ composer update --no-plugins
Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update
Generating autoload files
> symfony-cmd
'symfony-cmd' is not recognized as an internal or external command,
operable program or batch file.
Script symfony-cmd handling the auto-scripts event returned with error code 1
Script @auto-scripts was called via post-update-cmd

When composer is run without --no-plugins option, it doesn't:

λ composer update
"https://repo.packagist.org/packages.json" does not contain valid JSON
Parse error on line 1:

^
Expected one of: 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
https://repo.packagist.org could not be fully loaded, package information was loaded from the local cache and may be out of date
Loading composer repositories with package information
"https://repo.packagist.org/packages.json" does not contain valid JSON
Parse error on line 1:

We work behind a company HTTP Proxy, which is already set on env variables:

https_proxy=http://x.x.x.x:3128
http_proxy=http://x.x.x.x:3128

Full verbose of Composer update command:

λ composer -vvv update
Reading ./composer.json
Loading config file C:/Users/[my_username]/AppData/Roaming/Composer/config.json
Loading config file C:/Users/[my_username]/AppData/Roaming/Composer/auth.json
Loading config file ./composer.json
Checked CA file C:\Users\[my_username]\AppData\Local\Temp\opeB424.tmp: valid
Executing command (C:\Users\[my_username]\Documents\projects\[project_name]): git branch --no-color --no-abbrev -v
Reading C:/Users/[my_username]/AppData/Roaming/Composer/composer.json
Loading config file C:/Users/[my_username]/AppData/Roaming/Composer/config.json
Loading config file C:/Users/[my_username]/AppData/Roaming/Composer/auth.json
Loading config file C:/Users/[my_username]/AppData/Roaming/Composer/composer.json
Loading config file C:\Users\[my_username]\AppData\Roaming\Composer/auth.json
Reading C:\Users\[my_username]\AppData\Roaming\Composer/auth.json
Reading C:\Users\[my_username]\Documents\projects\[project_name]/vendor/composer/installed.json
Loading plugin PackageVersions\Installer
Loading plugin Symfony\Flex\Flex
Downloading https://repo.packagist.org/packages.json using proxy tcp://x.x.x.x:3128
Downloading https://repo.packagist.org/packages.json using proxy tcp://x.x.x.x:3128
Downloading https://repo.packagist.org/packages.json using proxy tcp://x.x.x.x:3128
Reading C:/Users/[my_username]/AppData/Local/Composer/repo/https---repo.packagist.org/packages.json from cache
"https://repo.packagist.org/packages.json" does not contain valid JSON
Parse error on line 1:

^
Expected one of: 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
https://repo.packagist.org could not be fully loaded, package information was loaded from the local cache and may be out of date

[nickforbizz]

RUN THIS ON YOUR TERMINAL

set http_proxy= set https_proxy=

[b-hayes]

This is happing on Ubuntu 20 (WSLv2) on Windows. Composer version 2.1.3 2021-06-09 16:31:20 PHP 7.4.3 (cli)

Setting up a new machine with all our microservices is nightmarishly slow atm.

[matthill33]

@b-hayes Same environment for me, same error happening, have had no luck with any suggestions above. PHP 7.4.3, Ubuntu WSL2 on Windows. Composer version 2.1.4

[engmalek]

i have the same problem also on PHP-8 version with creating laravel projects
then I try this composer global require laravel/installer then laravel new example-app and it is work fine

[haryandb]

Whetn I try php 7.2.14 is ok, 7.2.17 and 7.3.4 cause this error. Workaraound composer config disable-tls true

it is work for me