search

joomla / joomla-cms

Home of the Joomla! Content Management System
https://www.joomla.org
GNU General Public License v2.0
4.76k stars 3.65k forks source link

failed to open stream: Cannot connect to HTTPS server through proxy #19872

Closed aubreybox closed 6 years ago

aubreybox commented 6 years ago

Steps to reproduce the issue

Joomla! Extensions Directory™ (JED) now available with Install from Web on this page. By selecting "Add Install from Web tab" below, you agree to the JED Terms of Service and all applicable third party license terms.

Click that "Add Install from Web tab"

Expected result

Webinstaller is installed

Actual result

Error connecting to the server: fopen(https://downloads.joomla.org/extensions/install-from-web/1-1-1/plg_webinstaller_3.7v1.1.1.zip): failed to open stream: Cannot connect to HTTPS server through proxy

System information (as much as possible)

configuration.php:

    public $proxy_enable = '1';
    public $proxy_host = ip;
    public $proxy_port = port;

Version: 3.6.4

Additional comments

https_proxy="$ip:$port" wget https://downloads.joomla.org/extensions/install-from-web/1-1-1/plg_webinstaller_3.7v1.1.1.zip works

Same problem with joomla-updater

tonypartridge commented 6 years ago

First of all, please update to joomla! 3.8.5. We cannot do anything for a 3.6.4 version.

zero-24 commented 6 years ago

Version: 3.6.4

Please update the latest version 3.8.5 and try again. Please also check if install from a different https page like github works.

Please also check with the administrator of the proxy you use if that proxy supports https.

The other way arround would be to download that plugin using the browser and upload it as normal install :)

PhilETaylor commented 6 years ago

unable to replicate in 3.8.5

Steps -

I did however replicate an issue (which is a correct issue) when using Charles Proxy on mac to proxy ssl traffic - as the charles proxy ssl is a self-signed ssl - so quite rightly Joomla reported this and refused to relay secure traffic. This is expected behaviour.

so check your proxy is actually a valid proxy and has a valid ssl path

aubreybox commented 6 years ago

@tonypartridge I updated to 3.8.5 - didn't change anything

@PhilETaylor could be, I'm using my own proxy with self signed certificates; but that would not really explain why it works with wget. Except joomla (i.e. php) doesn't use the default systemwide CAs


1 <?php
2 $url="https://downloads.joomla.org/extensions/install-from-web/1-1-1/plg_webinstaller_3.7v1.1.1.zip";
3 $ctx = stream_context_create(['http' => ['proxy' => "tcp://$proxy:$port"],
4 'ssl' => ['capture_session_meta' => TRUE]]);
5 $html = file_get_contents($url , FALSE, $ctx);
6 $meta = stream_context_get_options($ctx)['ssl']['session_meta'];
7 var_dump($meta);
8 ?>

Results in:

PHP Warning:  file_get_contents(): Peer certificate CN=`*.s3-us-west-2.amazonaws.com' did not match expected CN=`downloads.joomla.org' in /tmp/test.php on line 5
PHP Warning:  file_get_contents(https://downloads.joomla.org/extensions/install-from-web/1-1-1/plg_webinstaller_3.7v1.1.1.zip): failed to open stream: Cannot connect to HTTPS server through proxy in /tmp/test.php on line 5

If I changed $url to "https://cdn.joomla.org/images/Joomla_logo.png", no error occurs.

PhilETaylor commented 6 years ago

Search @mbabker ‘s posts - this issue has already been covered to death. Google your new error message

Sent from my iPhone

On 10 Mar 2018, at 13:07, aubreybox notifications@github.com wrote:

@tonypartridge I updated to 3.8.5 - didn't change anything

@PhilETaylor could be, I'm using my own proxy with self signed certificates; but that would not really explain why it works with wget. Except joomla (i.e. php) doesn't use the default systemwide CAs

1 <?php 2 $url="https://downloads.joomla.org/extensions/install-from-web/1-1-1/plg_webinstaller_3.7v1.1.1.zip"; 3 $ctx = stream_context_create(['http' => ['proxy' => "tcp://$proxy:$port"], 4 'ssl' => ['capture_session_meta' => TRUE]]); 5 $html = file_get_contents($url , FALSE, $ctx); 6 $meta = stream_context_get_options($ctx)['ssl']['session_meta']; 7 var_dump($meta); 8 ?> Results in:

PHP Warning: file_get_contents(): Peer certificate CN=*.s3-us-west-2.amazonaws.com' did not match expected CN=downloads.joomla.org' in /tmp/test.php on line 5 PHP Warning: file_get_contents(https://downloads.joomla.org/extensions/install-from-web/1-1-1/plg_webinstaller_3.7v1.1.1.zip): failed to open stream: Cannot connect to HTTPS server through proxy in /tmp/test.php on line 5 If I changed $url to "https://cdn.joomla.org/images/Joomla_logo.png", no error occurs.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

aubreybox commented 6 years ago

If you mean this thread: https://issues.joomla.org/tracker/joomla-cms/9281

The error occurs due to missing SNI and old curl/openssl versions.

Thats not the case here:

curl --version curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

openssl version
OpenSSL 1.1.1-pre3-dev xx XXX xxxx

php -v PHP 7.0.22-0ubuntu0.16.04.1 (cli) ( NTS ) Copyright (c) 1997-2017 The PHP Group Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies with Zend OPcache v7.0.22-0ubuntu0.16.04.1, Copyright (c) 1999-2017, by Zend Technologies

Using curl also kind of works:

curl -v --proxy "$ip:$port" -s "https://downloads.joomla.org/extensions/install-from-web/1-1-1/plg_webinstaller_3.7v1.1.1.zip"
*   Trying $ip...
* Connected to $ip ($ip) port $port (#0)
* Establish HTTP proxy tunnel to downloads.joomla.org:443
> CONNECT downloads.joomla.org:443 HTTP/1.1
> Host: downloads.joomla.org:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* found 175 certificates in /etc/ssl/certs/ca-certificates.crt
* found 528 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
*        server certificate verification OK
*        server certificate status verification SKIPPED
*        common name: *.joomla.org (matched)
*        server certificate expiration date OK
*        server certificate activation date OK
*        certificate public key: RSA
*        certificate version: #3
*        subject: OU=Domain Control Validated,CN=*.joomla.org
*        start date: Fri, 09 Mar 2018 13:58:40 GMT
*        expire date: Thu, 11 Jun 2020 13:58:40 GMT
*        issuer: C=BE,O=GlobalSign nv-sa,CN=AlphaSSL CA - SHA256 - G2
*        compression: NULL
* ALPN, server accepted to use http/1.1
> GET /extensions/install-from-web/1-1-1/plg_webinstaller_3.7v1.1.1.zip HTTP/1.1
> Host: downloads.joomla.org
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 303 See Other
< Set-Cookie: 84dbf5a3d4c9dc46bc81af2046c45ae8=ditdimcgl1rga0jm4nr1pglku4; path=/; secure; HttpOnly
< Location: https://s3-us-west-2.amazonaws.com/joomla-official-downloads/joomladownloads/extensions/ifw/plg_webinstaller_3.7v1.1.1.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIZ6S3Q3YQHG57ZRA%2F20180310%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180310T141404Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=b92725e8932574fa6010eb97aa6f370625087a82d25c743c4e1928c8ddebd120
< Content-Type: text/html; charset=utf-8
< Expires: Wed, 17 Aug 2005 00:00:00 GMT
< Last-Modified: Sat, 10 Mar 2018 14:14:04 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Content-Length: 0
< Date: Sat, 10 Mar 2018 14:14:04 GMT
< Accept-Ranges: bytes
< Server: LiteSpeed
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Referrer-Policy: no-referrer-when-downgrade
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Content-Security-Policy-Report-Only: default-src 'self'; script-src 'self' 'unsafe-inline' https://*.google-analytics.com https://*.googletagmanager.com https://*.googleapis.com https://*.gstatic.com https://*.google.com https://*.joomla.org https://*.pingdom.net https://*.googleapis.com https://*.doubleclick.net https://*.buysellads.com; style-src 'self' 'unsafe-inline' https://*.joomla.org https://fonts.googleapis.com; connect-src 'self' https://*.joomla.org https://*.pingdom.net https://*.doubleclick.net https://*.google-analytics.com; frame-src 'self' https://*.google.com https://www.googletagmanager.com https://*.adfront.org; font-src 'self' https://fonts.gstatic.com https://*.joomla.org; img-src 'self' https://*.google-analytics.com https://*.joomla.org https://*.pingdom.net https://*.doubleclick.net https://*.gstatic.com https://*.google.com https://*.googleapis.com https://*.buysellads.com https://online.swagger.io https://*.adfront.org; report-uri https://community.joomla.org/scripts/csp-reporter.php?source=downloads.joomla.org
< Alt-Svc: quic=":443"; ma=2592000; v="35,37,38,39"
< Connection: close
< 
* Closing connection 0
curl -v --proxy "$ip:$port" -s "https://s3-us-west-2.amazonaws.com/joomla-official-downloads/joomladownloads/extensions/ifw/plg_webinstaller_3.7v1.1.1.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIZ6S3Q3YQHG57ZR
A%2F20180310%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180310T141404Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=b92725e8932574fa6010eb97aa6f370625087a82d25c743c4e1928c8ddebd120" -o plg_webinstaller_3.7v1.1.1.zip
*   Trying $ip...
* Connected to $ip ($ip) port $port (#0)
* Establish HTTP proxy tunnel to s3-us-west-2.amazonaws.com:443
> CONNECT s3-us-west-2.amazonaws.com:443 HTTP/1.1
> Host: s3-us-west-2.amazonaws.com:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* found 175 certificates in /etc/ssl/certs/ca-certificates.crt
* found 528 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
*        server certificate verification OK
*        server certificate status verification SKIPPED
*        common name: *.s3-us-west-2.amazonaws.com (matched)
*        server certificate expiration date OK
*        server certificate activation date OK
*        certificate public key: RSA
*        certificate version: #3
*        subject: C=US,ST=Washington,L=Seattle,O=Amazon.com Inc.,CN=*.s3-us-west-2.amazonaws.com
*        start date: Tue, 26 Sep 2017 00:00:00 GMT
*        expire date: Wed, 26 Sep 2018 12:00:00 GMT
*        issuer: C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert Baltimore CA-2 G2
*        compression: NULL
* ALPN, server did not agree to a protocol
> GET /joomla-official-downloads/joomladownloads/extensions/ifw/plg_webinstaller_3.7v1.1.1.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIZ6S3Q3YQHG57ZRA%2F20180310%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180310T141404Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=b92725e8932574fa6010eb97aa6f370625087a82d25c743c4e1928c8ddebd120 HTTP/1.1
> Host: s3-us-west-2.amazonaws.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< x-amz-id-2: OY+RBhs5ysx0FKkVJ/WwxEaEI9qFV8DgS2YfKEv3J4Zf/9RrPq5pLxA1XBx4ATx6ZveXCfD8hkI=
< x-amz-request-id: 2A998FFABE11E9E9
< Date: Sat, 10 Mar 2018 14:14:21 GMT
< Last-Modified: Sat, 03 Jun 2017 10:59:12 GMT
< ETag: "7a89f0095aa4a38db69f5945085967a0"
< Accept-Ranges: bytes
< Content-Type: application/zip
< Content-Length: 14050
< Server: AmazonS3
< 
{ [14050 bytes data]
* Connection #0 to host $ip left intact
PhilETaylor commented 6 years ago

This is not a Joomla issue as proved by your own test script.

Your root issue is "Peer certificate CN=*.s3-us-west-2.amazonaws.com' did not match expected CN=downloads.joomla.org' "

That message has been covered over and over again in these issues and forum posts and the resolution is always 100% the server infrastructure and not Joomla's core files (as proved by your test script)

If I changed $url to "https://cdn.joomla.org/images/Joomla_logo.png", no error occurs.

Correct because the CDN and the downloads site have different redirects and paths to the endpoints.

You showing the versions of curl/openssl/and PHP (CLI) helps nothing, as these dont relate the the versions your Joomla sites runs with, you could have different curl/openssl/php versions running when Joomla runs - you need to check your PHP Configuration in Joomla Admin to see what compiled versions of curl and openssl are running in which PHP version you are running.

The solution to all this will be a reconfiguration of your PHP.

aubreybox commented 6 years ago

That message has been covered over and over again in these issues and forum posts and the resolution is always 100% the server infrastructure and not Joomla's core files

So what? It's an error that occured in Joomla which is the reason for asking here. If it has been covered thousand times, I'd ask why a workaround isn't well documented. You invest more time in complaining about the fact that it has been answered somewhere than in solving the original problem.

Even now I don't see a solution for the problem. You just claim that someone solved it somewhere.

PhilETaylor commented 6 years ago

I give up. If you want to be selective in your reading of my posts and ignore facts that's up to you.

If you are not technical to understand that running commands at the command line, and the runtime php configuration can be 100% different... then again, you need to learn that.

You have failed to provide any information relating to Joomla.

Even now I don't see a solution for the problem. You just claim that someone solved it somewhere.

No, you just chose to ignore the solution, to dispute the facts, and provide non-relavant (command line) versions to justify your denial of the facts.

Provide the information from the Joomla System Information Page - which is the ACTUAL PHP CONFIGURATION that is running at runtime proving your versions are correct, or not. Im 100% certain when you finally accept this fact, and upgrade your versions compiled in PHP, you will fix this issue.

Many people have already spent considerable time investigating this issue, and identifying the root cause...

aubreybox commented 6 years ago

For me, the solution was installing php-curl.

PhilETaylor commented 6 years ago

For me, the solution was installing php-curl.

/facepalm ...

tonypartridge commented 6 years ago

😳

On 10 Mar 2018, 20:39 +0000, Phil Taylor notifications@github.com, wrote:

For me, the solution was installing php-curl. /facepalm ... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

PhilETaylor commented 6 years ago

As Per The Documentation:

tonypartridge commented 6 years ago

I’m sure I mentioned on Twitter, what is this thing called ‘Documentation’?

On 10 Mar 2018, 21:00 +0000, Phil Taylor notifications@github.com, wrote:

As Per The Documentation:

• https://docs.joomla.org/Technical_requirements • https://docs.joomla.org/Optional_Technical_Requirements

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

PhilETaylor commented 6 years ago

@aubreybox as your issue is resolved please close this Github Issue

joomla-cms-bot commented 6 years ago

Set to "closed" on behalf of @Quy by The JTracker Application at issues.joomla.org/joomla-cms/19872

c33s commented 6 years ago

just for reference if other people are getting here afer a google search:

i assume that internally file_get_contents is used here as fallback and curl as first way to connect.

it looks like file_get_contents has a bug which "saves" the first url it accesses and then pin the certificate it gets there. so all further calls are pinned to the first cert.

also see: https://github.com/consolidation/Robo/issues/714