The repo.magento.com server's HTTPS/SSL are Not Compatible with OS X

[astorm]

Steps to reproduce

1. Get an OS X machine running Yosemite and latest a version of PHP from http://php-osx.liip.ch/
2. Ensure latest composer is running with composer selfupdate
3. Run composer create-project --repository=https://repo.magento.com/ magento/project-community-edition

   Expected result

4. Composer Downloads Magento

   Actual result

5. Download fails with the composer error "The "https://repo.magento.com/packages.json" file could not be downloaded: Failed to enable crypto"
6. Developer silently weeps

---

Full composer exception below.

After some debugging in Composer, it appears that the SSL/HTTPS implementation on https://repo.magento.com is incompatible with OS X Yosemite's SSL/HTTPS encryption.

Simplifying the calls composer makes produces similar errors for repo.magento.com. i.e. this PHP script

$context = stream_context_create();
$contents = file_get_contents('https://repo.magento.com/packages.json', false, $context);

produces the error

PHP Warning: file_get_contents(https://repo.magento.com/packages.json): failed to open stream

I'm sure someone there is already thinking something something Apple's fault. However, this only happens with repo.magento.com. i.e., I can still update via the Packagist HTTPS server, and my PHP can download an HTTPS packagist packages.json. Again, breaking it down to a simple script -- this works without the failed crypto error.

<?php
$context = stream_context_create();
$contents = file_get_contents('https://packagist.org/packages.json', false, $context);

I realize this is a hard problem to track down, but if The World's Biggest Composer Repository (i.e. the de-facto standard) works with Apple's crypto, it seems like a reasonable expectation that Magento's should do this same.

"Use a VM" is a reasonable suggestion for a work around, but not an acceptable solution to this problem. "Use a mirror" is a reasonable suggestion for a work around, but not an acceptable solution to this problem.

The aforementioned composer exception traces follow.

[Composer\Downloader\TransportException]                                           
The "https://repo.magento.com/packages.json" file could not be downloaded: Failed  
to enable crypto                                                                  
failed to open stream: operation failed                                            

Exception trace:
() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Util/RemoteFilesystem.php:464
Composer\Util\RemoteFilesystem->get() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Util/RemoteFilesystem.php:101
Composer\Util\RemoteFilesystem->getContents() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Repository/ComposerRepository.php:645
Composer\Repository\ComposerRepository->fetchFile() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Repository/ComposerRepository.php:462
Composer\Repository\ComposerRepository->loadRootServerFile() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Repository/ComposerRepository.php:256
Composer\Repository\ComposerRepository->hasProviders() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/DependencyResolver/Pool.php:99
Composer\DependencyResolver\Pool->addRepository() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Command/CreateProjectCommand.php:287
Composer\Command\CreateProjectCommand->installRootPackage() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Command/CreateProjectCommand.php:156
Composer\Command\CreateProjectCommand->installProject() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Command/CreateProjectCommand.php:143
Composer\Command\CreateProjectCommand->execute() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/vendor/symfony/console/Command/Command.php:259
Symfony\Component\Console\Command\Command->run() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/vendor/symfony/console/Application.php:844
Symfony\Component\Console\Application->doRunCommand() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/vendor/symfony/console/Application.php:192
Symfony\Component\Console\Application->doRun() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Console/Application.php:227
Composer\Console\Application->doRun() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/vendor/symfony/console/Application.php:123
Symfony\Component\Console\Application->run() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Console/Application.php:102
Composer\Console\Application->run() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/bin/composer:43
require() at /Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar:24

create-project [-s|--stability STABILITY] [--prefer-source] [--prefer-dist] [--repository REPOSITORY] [--repository-url REPOSITORY-URL] [--dev] [--no-dev] [--no-custom-installers] [--no-scripts] [--no-progress] [--no-secure-http] [--keep-vcs] [--no-install] [--ignore-platform-reqs] [--] [<package>] [<directory>] [<version>]

[maksek]

Hi @astorm, just to check, have you installed openssl?

[astorm]

@maksek Yes.

[maksek]

Just did fresh install on my iMac (i used macports):

$ sw_vers ProductName: Mac OS X ProductVersion: 10.11.5 BuildVersion: 15F34

~/dev/www/magento2-composer-last$ composer.phar create-project --repository=https://repo.magento.com/ magento/project-community-edition
You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug
    Authentication required (repo.magento.com):                                                                                 
      Username: b61a6b44761799381efede7ce1ca9bfb
      Password: 
Do you want to store credentials for repo.magento.com in /Users/maksek/.composer/auth.json ? [Yn] Y
Installing magento/project-community-edition (2.1.0)
  - Installing magento/project-community-edition (2.1.0)
    Downloading: 100%         

It works fine. Doesn't look like repo issue.

[hostep]

Hi @astorm

It looks like they use a pretty modern ciphersuite and protocols on repo.magento.com, which only allows TLS 1.2 and 1.1 connections: https://www.ssllabs.com/ssltest/analyze.html?d=repo.magento.com&s=52.203.74.110&latest

There is a possibility that the OpenSSL version your php installation is linked against, doesn't support those protocols yet.

You could try running these commands to figure out if your OpenSSL is able to connect to Magento's composer repo:

$ openssl s_client -connect repo.magento.com:443 -tls1_2 # this one should connect
$ openssl s_client -connect repo.magento.com:443 -tls1_1 # this one should connect
$ openssl s_client -connect repo.magento.com:443 -tls1   # this one shouldn't connect

If the two first two command don't work on your system, then OpenSSL is most likely to blame.

After some researching, it looks like the built-in PHP version of OS X is linked against LibreSSL, instead of OpenSSL, which does support TLS 1.1 and 1.2. The OpenSSL version on OS X is really really old and doesn't support TLS 1.1 or 1.2. So that might explain it.

If you are somehow able to link your PHP installation against the LibreSSL version of OS X, then you might get it to work. Or somehow get a more recent version of OpenSSL on your machine (without overwriting the systems default) and linking it against your PHP installation.

So I guess the issue is in the PHP packages from http://php-osx.liip.ch/, maybe open a support ticket over there?

I myself always use Macports to install PHP versions on my OS X machines: https://www.macports.org/ports.php?by=name&substr=php7 And those get linked against the very latest version of OpenSSL (which Macports also installs)

But if you never worked with Macports before, it might be a bit scary and it might take you a while to figure it out and setup everything properly.

I hope this helps.

[24198]

@hostep

An issue has been reported for the old OpenSSL version at https://github.com/liip/php-osx/issues/175. Currently running into the same problems as @astorm, I've updated OpenSSL but it looks like the PHP version of liip's package is compiled with default OpenSSL of OS X. After running phpinfo(); on my current setup (which is in fact a version of liip's osx, php-5.6.21) I can see that the configure command has the option --with-openssl the only disadvantage is that it's pointing to /usr.

Snippet from phpinfo(); './configure' '--prefix=/usr/local/php5' '--with-apxs2=/usr/sbin/apxs' '--with-config-file-scan-dir=/usr/local/php5/php.d' '--with-libxml-dir=shared,/usr' '--with-openssl=/usr'

[maksek]

TLS 1.0 is disabled on repo.magento.com due security. Good summary is here - https://blog.varonis.com/ssl-and-tls-1-0-no-longer-acceptable-for-pci-compliance/

[astorm]

@maksek Re: "working for you" -- that reminds of the time, years ago, when I made the team I was on start putting a dollar in a jar anytime they said "Works on my machine" I'm guessing you're installing php where you're building or linking against a custom installed version of PHP? The packages I'm using, and others with the same problem are using, looks like its built using whatever version os SSL ships on OS standard. (http://php-osx.liip.ch/)

Re: TLS 1.0 -- I'm not sure I follow. Disabling TLS 1.0 on servers that process credit cards makes a whole lots of sense -- but we're talking about a composer repository that distributes source code. Is this really required, or is it just it just easier? If its really required, does that mean you're going to remove the source from GitHub.com?

[astorm]

@hostep Yeah, it's the packages on http://php-osx.liip.ch/ --- honestly, I've burned a few days on this, I'll just resort to work arounds for my own work rather than fight upstream on this. If Magento wants to move to a distribution model that blocks a stock OS X and a popular (possibly the most popular?) PHP distribution channel that's their choice. It sends a clear message though that they're looking to build a less open development ecosystem.

[maksek]

So, thanks for @hostep, @24198 for finding issue https://github.com/liip/php-osx/issues/175 on http://php-osx.liip.ch/ which @astorm uses. Since it is Magento 2 github, and the issue not related to Magento 2, @astorm any more comments?

[astorm]

@maksek While you have a situation here where you can easily point to technical issue that's not yours, your assessment that it's not related to Magento 2 is incorrect. With things setup as they are right now an unknown number (thousands? Tens of thousands?) of OS X can't access Magento 2 via composer.

That's horrible from a product point of view. For developers already working in the ecosystem this means making massive changes to how they work. For developers not working in the ecosystem it creates a horrible first impression that will keep an unknown percentage of them away.

If that's an acceptable tradeoff for your team, then you can close this ticket. If you're not comfortable with that, I'd recommend you product teams takes steps, lateral or otherwise, to fix it. Two suggestions are to provide a composer repository where these developers can access the source code, or by working with the upstream provider (http://php-osx.liip.ch/) themselves to get this fixed.

[maksek]

Thanks for suggestions @astorm, we will investigate them.

[24198]

@maksek so far I've seen two affected packages / setups where this issue applies. One is the php-osx liip package and the other one is MAMP. Both come with compiled php versions, they are compiled with the older OpenSSL 0.9.8x version. I've created a small overview with the php versions / affected setups:

PHP Version/Setup	PHP LIIP OSX	MAMP (PRO)
5.6.10	N/A	x
5.6.17	x	N/A
5.6.19	x	N/A
5.6.21	x	N/A
7.0.0	x	x
7.0.7	x	N/A

Hopefully this helps :)

Kind regards,

Jaimy

[chregu]

I'm working on a fix (for php-osx), looking good that this will be available soon

[benmarks]

Thanks @24198 & @chregu.

@astorm You (and others) should be informed that we are discussing the implications raised in this thread. These discussions are internal at the moment, but expect both detail and action from us in the very near future. I am curious to know if the new Liip package solves the problems for you.

[24198]

@benmarks new Liip package works like a charm! Thank you @chregu for updating the package, it works flawlessly!

[fooman]

Possibly related issue https://github.com/magento/magento2/issues/5275

[astorm]

@benmarks I grabbed the latest builds of Liip and they were able to download repo.magento.com over HTTPS without the aforementioned error. Thank you for working with the upstream providers on this and restoring everyone's "install Magento" buttons :)

[realchriswells]

Seems that there is still issues when running Magento 2 in MAMP Pro.

I've tried downloading the latest version in Brew, but for some reason my machine is still telling me that I've got OpenSSL 0.9.8.

MAMP Pro 4 apparently fixes this issue, but there doesn't seem to be a release date on this.

If anyone has a link to a guide for upgrading OpenSSL for MAMP then I would be very interested in reading it!

[24198]

@realchriswells at the company I'm working are having the same issue. So today we've implemented a temporary solution, which doesn't seem to have any side effects as far as I'm aware off.

So basically you install a package from liip osx and add it to your .bash_profile, follow the instructions ont there page,http://php-osx.liip.ch/ and you're good to go. This enables the php for you command line, but leaves the MAMP php untouched. This allows you to install and update your Magento through composer, but you can still leave your MAMP to run the environment. If you decide to run Magento 2.1, remember that MAMP only has a PHP package for 7.0.0, for which the support has been dropped in 2.1.

NOTE this is however just a temporary solution, it's dirty and cheap, but does work for the moment. This is until MAMP has a release date where they fix the issue with a new openssl version.

EDIT We've found an issue with the above temporary solution for MAMP (PRO). Apparently if you change the version for php in cli, you should hook up your mysql.sock from your command line php version to your MAMP mysql like so: pdo_mysql.default_socket=/Applications/MAMP/tmp/mysql/mysql.sock

Do this in your php.ini of the command line php version and you're good to go again. Also, don't forget to update your php memory_limit if you are using the tmp solution. Liip's osx php package has a default memory_limit of 256M, this could lead to memory exhausting in your terminal when running setup:static-content:deploy

Hope this helps,

Kind Regards,

Jaimy

[riconeitzel]

Hey Guys,

can you please check the following:

I removed

"repositories": [
        {
            "type": "composer",
            "url": "https://repo.magento.com/"
        }
    ],

from composer.json – it worked fine.

with these line in place I'll get the crypto-error.

What exactly the difference is, you ask?! I don't know – that's some black magic composer voodoo for me :D

Best, Rico

/cc: @benmarks + sherrierhode

[astorm]

@riconeitzel https://repo.magento.com/ is a composer repository. Its the composer repository where Magento keeps its source code. Magento recently tightened the security on https://repo.magento.com/ is very aggressive way, which will make PHP compiled against older versions of Open SLL (or equivalent Crypto libraries) not be able to talk to it.

So, when you remove those lines from your composer.json, the error goes away because composer and PHP are no longer trying to communicate with repo.magento.com. Unfortunately, since composer can't communicate with repo.magento.com this means you can't download your packages :)

The only fix right now is to update your version of PHP so its compiled against the latest Open SSL. How you go about this will depend on how wyou're running PHP.

[riconeitzel]

@astorm it's for sure downloading all that stuff … 

this means you can't download your packages

so if repo.magento.com is the only source, there MUST BE any kind of communication?!

[astorm]

@riconeitzel Afraid I don't quite follow what you're saying there @riconeitzel -- but a "how composer works discussion" sounds less like a discussion for here and more like a discussion for elsewhere (Stack Exchange (http://magento.stackexchange.com/), Magento Forum, any Magento Slacks you're a member of)

[riconeitzel]

looks like for what ever reason composer grabbed the files from a voodoo cache after remove these lines … so thanks @astorm for letting me know.

[hostep]

@riconeitzel: indeed, composer caches its downloaded files in ~/.composer/cache/

@24198: your MAMP workaround solution is a really bad idea, but you already mentioned that, just mentioning it again so people don't try this if they don't know what they are doing :)

[tanberry]

Thanks all, we have published a Technical Bulletin to address this issue.

http://devdocs.magento.com/guides/v2.1/release-notes/tech_bull_tls-repo.html

[ghost]

I had the same issue and got it downloading by installing the LIIP package.

Unfortunately the problem persists when running php bin/magento sampledata:deploy. Does anyone know if there are differences between running composer and php bin/magento?

[ghost]

Do you mean composer update and php bin/magento setup:upgrade? Those are completely different. setup:upgrade updates the database schema and data.

[ghost]

No, I mean php bin/magento sampledata:deploy.

More specifically I was wondering if there are differences in how the php bin/magento command handles the current bash environment in comparison to composer.

I've added /usr/local/php5/bin to PATH in my ~/.bash_profile and running which php produces /usr/local/php5/bin/php, so at least running php should use the LIIP package.

[klein0r]

Finally found a solution:

• I've updates OpenSSL on my Mac by using brew (version 1.0.2h): brew install openssl
• Linked the new version to use it as systems default: brew link --force openssl
• Then I've downloaded AMPPS (http://www.ampps.com/)
• Switched to PHP Version 5.6 inside of AMPPS
• Added the following entry in my php.ini: openssl.cafile=/usr/local/etc/openssl/cert.pem
• Now I'm using this version to update my local composer magento repositories
• Still running everything in MAMP (not using the AMPPS servers)

To make it a bit easier I've added an alias to my .bash_profile:

alias phpamps="/Applications/AMPPS/php-5.6/bin/php"

Now I can use:

phpamps composer.phar update

Hope this helps

[RUjmiak]

So there is still no solution for MAMP?

[realchriswells]

Only MAMP 4 when that is released.

There is a beta release of it, however I haven't tried it because I need a stable working environment and can't be fussing over bugs.

[klein0r]

@RUjmiak I've tried to recompile php within MAMP 3 with updated OpenSSL (1.0.2h) but it is quite hard and not that easy (since you need to add the sources, ...). I think you should use a workaround like I've described, try MAMP beta (which expires after some time and is not that fun to use for daily development) or wait until MAMP 4 is released (like @realchriswells said).

[simobenso]

I've created a small overview with the php versions / affected setups:

PHP Version/Setup PHP LIIP OSX MAMP (PRO) 5.6.10 N/A x 5.6.17 x N/A 5.6.19 x N/A 5.6.21 x N/A 7.0.0 x x 7.0.7 x N/A

@24198 thanks for this overview. Does this mean that this issue doesn't occur on MAMP 3.5.2 + PHP 7.0.7 ?

[24198]

@simobenso to be honest I don't know, at the time I created the overview, PHP 7.0.7 wasn't available and I have not tried it ever since. So I could be that I might work, but if I were to make an educated guess, it probably won't. Since it probably be compiled with the older version of openSSL, which is the real issue, on the other hand I might be completely wrong.

So to find the one true answer is only by trying, sorry I couldn't help more on a short notice. Hopefully you'll find the answer you're looking for.

Kind Regards,

Jaimy