croservices / cro-http

HTTP (including HTTPS and HTTP/2) support for the Cro library for building distributed systems in Raku.
https://cro.services/
Artistic License 2.0
49 stars 26 forks source link

Cro::HTTP::Client is not behaving correctly when using https proxy #185

Open Zer0-Tolerance opened 1 year ago

Zer0-Tolerance commented 1 year ago

Hi, I'm trying to use a proxy to debug a script and I need to an SSL intercepting proxy Burp Proxy but when doing testing I've discovered that Cro::HTTP::Client is actually doing an HTTP request on port 443 instead of a HTTPS one:

> use Cro::HTTP::Client
> my $ua = Cro::HTTP::Client.new(:cookie-jar);
> %*ENV{'HTTPS_PROXY'}='http://127.0.0.1:8080'
http://127.0.0.1:8080
> my $r=await $ua.get: 'https://www.bt.com';
An operation first awaited:
  in block <unit> at <unknown file> line 1

Died with the exception:
    Server responded with 400 Bad Request (GET http://127.0.0.1:8080)
      in block  at .rakubrew/versions/moar-2023.06/share/perl6/site/sources/3D7AE8DD442BE31392D93ECFBF0B6CACEC0825D6 (Cro::HTTP::Client) line 676

Do you have any idea why it's behaving this way ?

Zer0-Tolerance commented 1 year ago

Just found another bug with query params , they are duplicated when using the proxy. without http proxy => works fine

[2] >  %*ENV{'HTTPS_PROXY'}=Nil
[4] > await Cro::HTTP::Client.new.get: 'https://www.google.com',query => {q => 'test.com'};
Cro::HTTP::Response.new(request => Cro::HTTP::Request.new(body-parser-selector => Cro::HTTP::BodyParserSelector::RequestDefault, body-serializer-selector => Cro::HTTP::BodySerializerSelector::RequestDefault, connection => Any, method => "GET", target => "/?q=test.com", original-target => Str, auth => Any, annotations => {}, http-version => "2", http2-stream-id => 1), status => 200, body-parser-selector => Cro::HTTP::BodyParserSelector::ResponseDefault, body-serializer-selector => Cro::HTTP::BodySerializerSelector::ResponseDefault, cancellation-vow => Promise::Vow.new(promise => Promise.new(scheduler => ThreadPoolScheduler.new(uncaught_handler => -> $exception { #`(Block|2954449550600) ... }), status => PromiseStatus::Planned)), http-version => "2.0", http2-stream-id => 1)

with http proxy

[5] >  %*ENV{'HTTPS_PROXY'}='http://127.0.0.1:8080'
http://127.0.0.1:8080
[6] > await Cro::HTTP::Client.new.get: 'https://www.google.com',query => {q => 'test.com'};
An operation first awaited:
  in block <unit> at <unknown file> line 1

Died with the exception:
    Server responded with 400 Bad Request (GET http://127.0.0.1:8080)
      in block  at .rakubrew/versions/moar-2023.06/share/perl6/site/sources/3D7AE8DD442BE31392D93ECFBF0B6CACEC0825D6 (Cro::HTTP::Client) line 676

actual request sent is [00:25:05.389][127.0.0.1:52368] 127.0.0.1:52368: Bad HTTP request line: b'GET https://www.google.com?q=test.com?q=test.com HTTP/1.1' with ?q=test.com?q=test.com being the bogus part.