Compatibility issues HTTP_PROXY returns `HTTP/1.0 200 Connection established`

shenyi0828 commented 12 months ago

Package

http2_adapter

Version

1.2.4

Operating-System

Android, iOS, Web, MacOS, Linux, Windows

Output of `flutter doctor -v`

No response

Dart Version

No response

Steps to Reproduce

Use Surge as System Proxy (HTTP)

Expected Result

Request as expected.

Actual Result

error: Proxy cannot be initialized

Good Example:

CONNECT www.baidu.com:443 HTTP/1.1
Host: www.baidu.com:443
User-Agent: curl/8.1.1
Proxy-Connection: Keep-Alive

HTTP/1.0 200 Connection established

https://www.rfc-editor.org/rfc/rfc9110#section-9.3.6

RFC metions a 200 response is good . but code is

if (statusLine.startsWith('HTTP/1.1 200')) {
          completerProxyInitialization.complete();
        } else {
          completerProxyInitialization.completeError(
            SocketException('Proxy cannot be initialized'),
          );
        }

Also Proxy-Connection: Keep-Alive is good for proxy

AlexV525 commented 12 months ago

Could you set a debug breakpoint at L207 and inspect the result of statusLine?

shenyi0828 commented 12 months ago

Sorry, I'm debugging this issue by just sending same content through PacketSender, which can be accquire from https://packetsender.com/. And getting response from my Proxy Server is like below

shenyi0828 commented 12 months ago

I'm following code from this source

~~P.S. tripple \r\n is required to get response from Surge (Proxy Server ) , one is for line break , two are for ending HEADERS frame~~

shenyi0828 commented 12 months ago

curl works with my proxy , and I'm getting this verbose logs :

domain below is replaced by 'foo.com'

* Establish HTTP proxy tunnel to foo.com:443
> CONNECT foo.com:443 HTTP/1.1
> Host: foo.com:443
> User-Agent: curl/8.1.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 200 Connection established
<
* CONNECT phase completed
* CONNECT tunnel established, response 200
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /Users/XXX/XXX/ssl/cacert.pem
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.foo.com
*  start date: Sep 26 00:00:00 2023 GMT
*  expire date: Dec 25 23:59:59 2023 GMT
*  subjectAltName: host "foo.com" matched cert's "*.foo.com"
*  issuer: C=AT; O=ZeroSSL; CN=ZeroSSL RSA Domain Secure Site CA
*  SSL certificate verify ok.
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: foo.com]
* h2 [:path: /badge/aaad37f232c0/123.png]
* h2 [user-agent: curl/8.1.1]
* h2 [accept: */*]
* Using Stream ID: 1 (easy handle 0x15a80bc00)
> GET /badge/aaad37f232c0/2a9dfc9c3bc8c50099877013d0c524a1.png HTTP/2
> Host: foo.com
> User-Agent: curl/8.1.1
> Accept: */*
>
< HTTP/2 200
< last-modified: Wed, 01 Nov 2023 08:39:52 GMT
< etag: "2a9dfc9c3bc8c50099877013d0c524a1"
< content-type: image/png
< date: Mon, 04 Dec 2023 07:31:32 GMT
< server: tencent-cos
< vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
< x-cos-hash-crc64ecma: 12697382968711187629316
< x-cos-meta-md5: 111
< x-cos-request-id: 111=
< content-length: 80438
< accept-ranges: bytes
< x-nws-log-uuid: 111
< x-cache-lookup: Cache Hit
< access-control-allow-methods: OPTIONS,HEAD,GET
<

shenyi0828 commented 12 months ago

Charles returns

$ nc -4  10.251.41.82 8888
CONNECT www.baidu.com:443 HTTP/1.1
Host:www.baidu.com:443
Proxy-Connection: Keep-Alive

HTTP/1.1 200 Connection established

~~Note that only double CRLF is required .~~

shenyi0828 commented 12 months ago

Heres Surge returns

$ nc -4  10.251.41.82 6152
CONNECT www.baidu.com:443 HTTP/1.1
Host:www.baidu.com:443
Proxy-Connection: Keep-Alive

HTTP/1.0 200 Connection established

AlexV525 commented 12 months ago

Could you set a debug breakpoint at L207 and inspect the result of statusLine?

shenyi0828 commented 12 months ago

Could you set a debug breakpoint at L207 and inspect the result of statusLine?

AlexV525 commented 11 months ago

@shenyi0828 Where did you get the Surge proxy from? I'm trying to setup an HTTP/1.0 environment for testing.

shenyi0828 commented 11 months ago

@shenyi0828 Where did you get the Surge proxy from? I'm trying to setup an HTTP/1.0 environment for testing.

https://nssurge.com/

kuhnroyal commented 11 months ago

@shenyi0828 Can you test & verify the linked PR?

shenyi0828 commented 11 months ago

@shenyi0828 Can you test & verify the linked PR?

Sorry, I'm not familiar with testing lib from source code. You can just install the Surge App , which can downloaded from https://nssurge.com/ , and then go to Overview -> LAN DEVICE TAKEOVER -> turn on "HTTP & SOCKS5 Proxy", will show ip:port -> setup mobile phone network system proxy to the ip:port got from previous step.

AlexV525 commented 11 months ago

Sorry, I'm not familiar with testing lib from source code.

@shenyi0828 Write this in your pubspec:

dependencies:
  dio_http2_adapter:
    git:
      url: https://github.com/cfug/dio
      ref: 'fix/major-version-protocols'
      path: plugins/http2_adapter

shenyi0828 commented 11 months ago

Sorry, I'm not familiar with testing lib from source code.

@shenyi0828 Write this in your pubspec:
dependencies:
  dio_http2_adapter:
    git:
      url: https://github.com/cfug/dio
      ref: 'fix/major-version-protocols'
      path: plugins/http2_adapter

Connection is established. But handshake failed.

<2023-12-20T14:20:03.557236> null: DioException [unknown]: null
         Error: HandshakeException: Connection terminated during handshake
           ERROR: null: DioException [unknown]: null

AlexV525 commented 11 months ago

I'd rather think it's the proxy issue than the adapter.

shenyi0828 commented 11 months ago

I'd rather think it's the proxy issue than the adapter.

proxy works fine with curl

https_proxy=127.0.0.1:6152 curl --http2 -v https://xxx

got

* Uses proxy env variable https_proxy == '127.0.0.1:6152'
*   Trying 127.0.0.1:6152...
* Connected to 127.0.0.1 (127.0.0.1) port 6152 (#0)
* CONNECT tunnel: HTTP/1.1 negotiated
* allocate connect buffer
* Establish HTTP proxy tunnel to yuncdn.xxx.com:443
> CONNECT yuncdn.xxx.com:443 HTTP/1.1
> Host: yuncdn.xxx.com:443
> User-Agent: curl/8.1.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 200 Connection established
<
* CONNECT phase completed
* CONNECT tunnel established, response 200
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /Users/xxx/anaconda3/ssl/cacert.pem
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.xxx.com
*  start date: Nov 24 00:00:00 2023 GMT
*  expire date: Feb 22 23:59:59 2024 GMT
*  subjectAltName: host "yuncdn.xxx.com" matched cert's "*.xxx.com"
*  issuer: C=AT; O=ZeroSSL; CN=ZeroSSL RSA Domain Secure Site CA
*  SSL certificate verify ok.
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: yuncdn.xxx.com]
* h2 [:path: /badge/aaad37f232c0/xxx.png]
* h2 [user-agent: curl/8.1.1]
* h2 [accept: */*]
* Using Stream ID: 1 (easy handle 0x142814e00)

shenyi0828 commented 11 months ago

by the way, another http2_adapter issue is that if backend DO NOT support HTTP/2 . Request will fail with error, on the contrast, curl will success with HTTP 1.1 .

AlexV525 commented 11 months ago

Connection is established. But handshake failed.

<2023-12-20T14:20:03.557236> null: DioException [unknown]: null
         Error: HandshakeException: Connection terminated during handshake
           ERROR: null: DioException [unknown]: null

I cannot reproduce the issue locally. Can you provide a reproducible URL?

logs:

*** Request ***
uri: https://pub.dev/?xx=6
method: GET
responseType: ResponseType.plain
followRedirects: true
persistentConnection: true
connectTimeout: null
sendTimeout: null
receiveTimeout: null
receiveDataWhenStatusError: true
extra: {}
headers:

*** Response ***
uri: https://pub.dev/?xx=6
statusCode: 200
headers:
 date: Fri, 22 Dec 2023 02:37:36 GMT
 content-type: text/html; charset="utf-8"
 content-length: 30554
 vary: Accept-Encoding
 x-powered-by: Dart with package:shelf
 cache-control: public, max-age=0
 strict-transport-security: max-age=31536000; preload
 referrer-policy: no-referrer-when-downgrade
 x-frame-options: deny
 x-xss-protection: 1; mode=block
 x-content-type-options: nosniff
 content-security-policy: default-src 'self' https:;font-src 'self' data: https://fonts.googleapis.com/ https://fonts.gstatic.com/;img-src 'self' https: data:;manifest-src 'none';object-src 'none';script-src 'self' https://tagmanager.google.com https://www.googletagmanager.com/ https://www.google.com/ https://www.google-analytics.com/ https://ssl.google-analytics.com https://adservice.google.com/ https://ajax.googleapis.com/ https://apis.google.com/ https://unpkg.com/ https://www.gstatic.com/ https://gstatic.com https://accounts.google.com/gsi/client;style-src 'self' https://unpkg.com/ https://pub.dartlang.org/static/ 'unsafe-inline' https://fonts.googleapis.com/ https://gstatic.com https://www.gstatic.com/ https://tagmanager.google.com https://accounts.google.com/gsi/style
 via: 1.1 google

cfug / dio