croservices / cro-http

HTTP (including HTTPS and HTTP/2) support for the Cro library for building distributed systems in Raku.
https://cro.services/
Artistic License 2.0
49 stars 26 forks source link

Cro::HTTP::Client hangs with some HTTPS servers #45

Closed shalomb closed 4 years ago

shalomb commented 6 years ago

Following the documentation at https://cro.services/docs/reference/cro-http-client, I set out with a simple script to test some REST API interactions against https://vial-http.appspot.com/

#!/usr/bin/env perl6
my $resp = await Cro::HTTP::Client.get('https://vial-http.appspot.com');
say $resp;

However, this hangs almost indefinitely and the issue is reproducible every time.

Trying out a few different other URLs, I notice this seems to be correlated with HTTPS URLs (with the problem seemingly prevalent with those associated with google? but not always).

while read -r url; do ( set -x; timeout 4 perl6 ./cro.p6 "$url" &>/dev/null ); echo $?; done < urls                                                                            
+ timeout 4 perl6 ./cro.p6 https://addons.mozilla.org/en-GB/firefox/                                                                                                                                                
0                                                                                                                                                                                                                   
+ timeout 4 perl6 ./cro.p6 'https://isbnsearch.org/search?s=Think+Perl+6'                                                                                                                                           
1                                                                                                                                                                                                                   
+ timeout 4 perl6 ./cro.p6 https://cro.services/docs/reference/cro-http-client                                                                                                                                      
0                                                                                                                                                                                                                   
+ timeout 4 perl6 ./cro.p6 'https://github.com/croservices/cro-http/issues?utf8=%E2%9C%93&q=is%3Aissue+https'                                                                                                       
0                                                                                                                                                                                                                   
+ timeout 4 perl6 ./cro.p6 https://ipapi.co/                                                                                                                                                                        
0                                                                                                                                                                                                                   
+ timeout 4 perl6 ./cro.p6 https://www.yahoo.com/                                                                                                                                                                   
124                                                                                                                                                                                                                 
+ timeout 4 perl6 ./cro.p6 'https://twitter.com/?lang=en'                                                                                                                                                           
1                                                                                                                                                                                                                   
+ timeout 4 perl6 ./cro.p6 https://www.facebook.com/                                                                                                                                                                
0                                                                                                                                                                                                                   
+ timeout 4 perl6 ./cro.p6 https://en.wikipedia.org/wiki/Facebook                                                                                                                                                   
124                                                                                                                                                                                                                 
+ timeout 4 perl6 ./cro.p6 'https://www.instagram.com/?hl=en'                                                                                                                                                       
0     
+ timeout 4 perl6 ./cro.p6 https://google.co.uk/                                                                                                                                                         [196/16140]
124                                                                                                                                                                                                                
+ timeout 4 perl6 ./cro.p6 http://vial-http.appspot.com/                                                                                                                                                           
0
+ timeout 4 perl6 ./cro.p6 https://vial-http.appspot.com/                                                                                                                                                          
124
+ timeout 4 perl6 ./cro.p6 http://google.com/
0                                                                                                                                                                                                                  
+ timeout 4 perl6 ./cro.p6 https://google.com/
124                                                                                                                                                                                                                
+ timeout 4 perl6 ./cro.p6 http://google.com.au/                                                                                                                                                                   
0                                                                                                                                                                                                                  
+ timeout 4 perl6 ./cro.p6 https://google.com.au/
124
+ timeout 4 perl6 ./cro.p6 https://cloud.google.com/                                                                                                                                                               
124
+ timeout 4 perl6 ./cro.p6 https://www.google.com/gmail/
124
+ timeout 4 perl6 ./cro.p6 https://accounts.google.com/ServiceLogin
124                                                                                                                                                                                                                
+ timeout 4 perl6 ./cro.p6 https://itunes.apple.com/us/
0
+ timeout 4 perl6 ./cro.p6 https://www.blog.google/products                                                                                                                                                        
124
+ timeout 4 perl6 ./cro.p6 https://zapier.com/apps/                                                                                                                                                                
124                                                                                                                                                                                                                
+ timeout 4 perl6 ./cro.p6 https://www.dropbox.com/
0                                                                                                                                                                                                                  
+ timeout 4 perl6 ./cro.p6 https://passwords.google.com/
124                                                                                                                                                                                                                
+ timeout 4 perl6 ./cro.p6 https://images.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://google.com/settings/ads
124
+ timeout 4 perl6 ./cro.p6 https://firebase.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://apps.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://mail.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://gmail.com/
124
+ timeout 4 perl6 ./cro.p6 https://myaccount.google.com/security
124
+ timeout 4 perl6 ./cro.p6 https://translate.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://books.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://fonts.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://domains.google/
124
+ timeout 4 perl6 ./cro.p6 https://support.google.com/legal
124
+ timeout 4 perl6 ./cro.p6 http://history.google.com/
124
$ cat /etc/os-release                                                                                                                                                             
PRETTY_NAME="Debian GNU/Linux buster/sid"                                                                                                                                                                          
NAME="Debian GNU/Linux"                                                                                                                                                                                            
ID=debian                                                                                                                                                                                                          
HOME_URL="https://www.debian.org/"                                                                                                                                                                                 
SUPPORT_URL="https://www.debian.org/support"                                                                                                                                                                       
BUG_REPORT_URL="https://bugs.debian.org/" 

$ perl6 -v                                                                                                                                                                        
This is Rakudo Star version 2018.04.1 built on MoarVM version 2018.04.1                                                                                                                                            
implementing Perl 6.c.  

$ zef info Cro::HTTP::Client
- Info for: Cro::HTTP::Client
- Identity: Cro::HTTP:ver<0.7.6.1>
- Recommended By: /usr/share/perl6/site
- Installed: Yes
Description:     Asynchronous HTTP, both client and server side. Includes HTTP/2.0 support.
License:         Artistic-2.0
Source-url:      https://github.com/croservices/cro-http.git
Provides: 47 modules
Depends: 11 items

$ dpkg -l | grep -i ssl                                                                                                                                                           
ii  libcrypt-openssl-bignum-perl                    0.09-1                       amd64                        Perl module to access OpenSSL multiprecision integer arithmetic libraries                            
ii  libcrypt-openssl-random-perl                    0.11-1+b5                    amd64                        module to access the OpenSSL pseudo-random number generator                                          
ii  libcrypt-openssl-rsa-perl                       0.30-1                       amd64                        module for RSA encryption using OpenSSL                                                              
ii  libcurl4:amd64                                  7.60.0-2                     amd64                        easy-to-use client-side URL transfer library (OpenSSL flavour)                                       
ii  libflac8:amd64                                  1.3.2-3                      amd64                        Free Lossless Audio Codec - runtime C library                                                        
ii  libio-socket-ssl-perl                           2.058-1                      all                          Perl module implementing object oriented interface to SSL sockets                                    
ii  libnet-ssleay-perl                              1.85-1                       amd64                        Perl module for Secure Sockets Layer (SSL)                                                           
ii  libssl-dev:amd64                                1.1.0h-4                     amd64                        Secure Sockets Layer toolkit - development files                                                     
ii  libssl1.0.0:amd64                               1.0.1t-1+deb8u6              amd64                        Secure Sockets Layer toolkit - shared libraries                                                      
ii  libssl1.0.2:amd64                               1.0.2o-1                     amd64                        Secure Sockets Layer toolkit - shared libraries                                                      
ii  libssl1.1:amd64                                 1.1.0h-4                     amd64                        Secure Sockets Layer toolkit - shared libraries                                                      
ii  libwavpack1:amd64                               5.1.0-3                      amd64                        audio codec (lossy and lossless) - library                                                           
ii  libzstd1:amd64                                  1.3.4+dfsg-3                 amd64                        fast lossless compression algorithm                                                                  
ii  openfortivpn                                    1.7.1-1                      amd64                        Fortinet client for PPP+SSL VPN tunnel services                                                      
ii  openssl                                         1.1.0h-4                     amd64                        Secure Sockets Layer toolkit - cryptographic utility                                                 
ii  perl-openssl-defaults:amd64                     3                            amd64                        version compatibility baseline for Perl OpenSSL packages                                             
ii  python-certifi                                  2018.4.16-1                  all                          root certificates for validating SSL certs and verifying TLS hosts                                   
ii  python3-certifi                                 2018.4.16-1                  all                          root certificates for validating SSL certs and verifying TLS hosts (python3)                         
ii  ssl-cert                                        1.0.39                       all                          simple debconf wrapper for OpenSSL   
jnthn commented 6 years ago

Please could you try it with:

my $resp = await Cro::HTTP::Client.get('https://vial-http.appspot.com', :http<1.1>);
shalomb commented 6 years ago

That was super quick @jnthn :)

Indeed, dropping down to HTTP 1.1 gets it working ..

 $ perl6 ./cro-http-client.p6 'https://vial-http.appspot.com'                                                                                                                                  
HTTP/1.1 200 OK                                                                                                                                                                                                    
Content-Type: application/json                                                                                                                                                                                     
X-Cloud-Trace-Context: 2c39fa66ee4fb9a6fb3a36d04e93a683;o=1                                                                                                                                                        
Date: Thu, 26 Jul 2018 23:19:07 GMT                                                                                                                                                                                
Server: Google Frontend                                                                                                                                                                                            
Content-Length: 19                                                                                                                                                                                                 
Alt-Svc: quic=":443"; ma=2592000; v="44,43,39,35"                                                                                                                                                                  
Connection: close  

I shall try and gather a strace too.

shalomb commented 6 years ago

strace-cro-http-client-vial-http.appspot.com.txt

jnthn commented 6 years ago

OK, that means it's something related to our HTTP/2.0 implementation. Alas, I don't have time to investigate right now (about to do some travel); @Altai-man perhaps you'd be able to reproduce/investigate this a bit sooner than I can?

In the meantime, I guess the :http<1.1> workaround helps. If making many requests, then it can be more convenient to make an instance of Cro::HTTP::Client and pass that option in the constructor, rather than having to pass it on each request.

shalomb commented 6 years ago

Thanks @jnthn for the good work with cro .. :+1: Safe travels!!

@Altai-man - do let me know if I can assist with repros, additional information .. happy to help.

Altai-man commented 6 years ago

Hangs are not my strongest point, but let's try to resolve this.

Altai-man commented 6 years ago

It consists of two parts:

Altai-man commented 6 years ago

Second part is addressed in https://github.com/croservices/cro-http/commit/3fbb06b609f9f20fa3209c569c90cc2e3049389e As for first one, I suspect we are missing either some negotiation scheme or something like that, or we need to run our client implementation against spec debug tool to see if we're missing something.

ohmycloud commented 5 years ago

Unknown string encoding: 'gb2312':

use Cro::HTTP::Client;
my $resp = await Cro::HTTP::Client.get('https://translate.google.cn', :http<1.1>);
say await $resp.body-text();

Output:

An operation first awaited:
  in block <unit> at t1.pl6 line 32
Died with the exception:
    Unknown string encoding: 'gb2312'
      in block  at /Applications/Rakudo/share/perl6/site/sources/557D6C932894CB1ADE0F83C0596851F9212C2A67 (Cro::MessageWithBody) line 51

Perl 6

perl6 -v
This is Rakudo Star version 2018.10 built on MoarVM version 2018.10
implementing Perl 6.c.

Cro::HTTP

zef info Cro::HTTP::Client
- Info for: Cro::HTTP::Client
- Identity: Cro::HTTP:ver<0.8.0>
- Recommended By: /Applications/Rakudo/share/perl6/site
- Installed: Yes
Description:     Asynchronous HTTP, both client and server side. Includes HTTP/2.0 support.
License:     Artistic-2.0
Source-url:  https://github.com/croservices/cro-http.git
Provides: 47 modules
Depends: 11 items
Altai-man commented 5 years ago

Judging by https://github.com/perl-gsoc-2019/ideas/blob/master/perl6/GB2312-encoding-decoding.md it sounds like this encoding is not implemented at all on MoarVM level, so we really cannot help it from Cro level.

MorayJ commented 5 years ago

Please could you try it with:

my $resp = await Cro::HTTP::Client.get('https://vial-http.appspot.com', :http<1.1>);

I'm getting hanging and (max cpu) issue with your documentation example

my $client = Cro::HTTP::Client.new( headers => [ User-agent => 'Cro' ]); my $resp = await $client.get('https://www.perl6.org/');

It's fixed by adding :http<1.1>, so if this is the same thing and can't be fixed would suggest updating the documentation.

Altai-man commented 5 years ago

@MorayJ there was a known hang with HTTP/2 parser, it was fixed on master recently, but not yet released. Can you try it with fresh clone of cro-http repo? Using latest I can't reproduce the hang.

MorayJ commented 5 years ago

This is what I did, which I think is what you're wanting git clone https://github.com/croservices/cro-http.git cd cro-http/ zef install Cro::HTTP . --force-install

Still hanging...

Altai-man commented 5 years ago

If you are installing a distribution from the local directory, just zef install . will be enough, possibly with --force-install because you are installing a module with the same version as you might have installing.

MorayJ commented 5 years ago

I think I've tried it ok that way now. It doesn't use up all the cpu, but it still seems to be hanging for me

CurtTilmes commented 4 years ago

Just another voice -- I experienced the hanging and came here looking for a solution. The :http<1.1> fixes it for me.