search

AdguardTeam / gomitmproxy

Simple golang mitm proxy implementation
https://adguard.com/
GNU General Public License v3.0
293 stars 59 forks source link

auth connection should not be closed #19

Closed airycanon closed 1 year ago

airycanon commented 1 year ago

before this fix

client log by git clone

$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 HTTPS_PROXY=http://abc:def@127.0.0.1:8080 git clone https://<hidden host here>/root/e2eproject

20:53:19.445918 git.c:460               trace: built-in: git clone https://<hidden host here>/root/e2eproject
正克隆到 'e2eproject'...
20:53:19.455319 run-command.c:655       trace: run_command: git remote-https origin https://<hidden host here>/root/e2eproject
20:53:19.461420 git.c:750               trace: exec: git-remote-https origin https://<hidden host here>/root/e2eproject
20:53:19.461839 run-command.c:655       trace: run_command: git-remote-https origin https://<hidden host here>/root/e2eproject
20:53:19.468176 http.c:701              == Info: Couldn't find host <hidden host here> in the (nil) file; using defaults
20:53:19.471605 http.c:701              == Info:   Trying 127.0.0.1:8080...
20:53:19.525761 http.c:701              == Info: Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
20:53:19.525800 http.c:701              == Info: allocate connect buffer
20:53:19.525804 http.c:701              == Info: Establish HTTP proxy tunnel to <hidden host here>:443
20:53:19.525839 http.c:648              => Send header, 0000000137 bytes (0x00000089)
20:53:19.525844 http.c:660              => Send header: CONNECT <hidden host here>:443 HTTP/1.1
20:53:19.525847 http.c:660              => Send header: Host: <hidden host here>:443
20:53:19.525849 http.c:660              => Send header: User-Agent: git/2.39.1
20:53:19.525851 http.c:660              => Send header: Proxy-Connection: Keep-Alive
20:53:19.525853 http.c:660              => Send header:
20:53:19.580981 http.c:648              <= Recv header, 0000000044 bytes (0x0000002c)
20:53:19.581008 http.c:660              <= Recv header: HTTP/1.1 407 Proxy Authentication Required
20:53:19.581047 http.c:648              <= Recv header, 0000000027 bytes (0x0000001b)
20:53:19.581050 http.c:660              <= Recv header: Proxy-Authenticate: Basic
20:53:19.581076 http.c:648              <= Recv header, 0000000019 bytes (0x00000013)
20:53:19.581079 http.c:660              <= Recv header: Content-Length: 0
20:53:19.581090 http.c:648              <= Recv header, 0000000002 bytes (0x00000002)
20:53:19.581093 http.c:660              <= Recv header:
20:53:19.581100 http.c:701              == Info: Establish HTTP proxy tunnel to <hidden host here>:443
20:53:19.581109 http.c:701              == Info: Proxy auth using Basic with user 'abc'
20:53:19.581148 http.c:648              => Send header, 0000000178 bytes (0x000000b2)
20:53:19.581160 http.c:660              => Send header: CONNECT <hidden host here>:443 HTTP/1.1
20:53:19.581163 http.c:660              => Send header: Host: <hidden host here>:443
20:53:19.581166 http.c:660              => Send header: Proxy-Authorization: Basic <redacted>
20:53:19.581169 http.c:660              => Send header: User-Agent: git/2.39.1
20:53:19.581172 http.c:660              => Send header: Proxy-Connection: Keep-Alive
20:53:19.581174 http.c:660              => Send header:
20:53:19.581182 http.c:701              == Info: Proxy CONNECT aborted
20:53:19.581197 http.c:701              == Info: Closing connection 0
20:53:19.581265 run-command.c:655       trace: run_command: 'git credential-osxkeychain erase'
20:53:19.601879 git.c:750               trace: exec: git-credential-osxkeychain erase
20:53:19.602415 run-command.c:655       trace: run_command: git-credential-osxkeychain erase

proxy server log:

2023/05/31 20:53:10 96413#6 [info] start listening to [::]:8080
2023/05/31 20:53:19 96413#6 [debug] id=100001: accepted connection from 127.0.0.1:63985
2023/05/31 20:53:19 96413#33 [debug] id=100001: waiting for request
2023/05/31 20:53:19 96413#33 [debug] id=100001-1: handle request CONNECT <hidden host here>:443
2023/05/31 20:53:19 96413#33 [debug] id=100001-1: proxy auth required
2023/05/31 20:53:19 96413#33 [debug] id=100001: closing connection due to: closing connection

After this fix

client log by git clone

$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 HTTPS_PROXY=http://abc:def@127.0.0.1:8080 git clone https://<hidden host here>/root/e2eproject

21:04:15.923638 git.c:460               trace: built-in: git clone https://<hidden host here>/root/e2eproject
正克隆到 'e2eproject'...
21:04:15.934876 run-command.c:655       trace: run_command: git remote-https origin https://<hidden host here>/root/e2eproject
21:04:15.940869 git.c:750               trace: exec: git-remote-https origin https://<hidden host here>/root/e2eproject
21:04:15.941332 run-command.c:655       trace: run_command: git-remote-https origin https://<hidden host here>/root/e2eproject
21:04:15.950524 http.c:701              == Info: Couldn't find host <hidden host here> in the (nil) file; using defaults
21:04:15.954399 http.c:701              == Info:   Trying 127.0.0.1:8080...
21:04:16.009533 http.c:701              == Info: Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
21:04:16.009548 http.c:701              == Info: allocate connect buffer
21:04:16.009551 http.c:701              == Info: Establish HTTP proxy tunnel to <hidden host here>:443
21:04:16.009584 http.c:648              => Send header, 0000000137 bytes (0x00000089)
21:04:16.009588 http.c:660              => Send header: CONNECT <hidden host here>:443 HTTP/1.1
21:04:16.009590 http.c:660              => Send header: Host: <hidden host here>:443
21:04:16.009592 http.c:660              => Send header: User-Agent: git/2.39.1
21:04:16.009594 http.c:660              => Send header: Proxy-Connection: Keep-Alive
21:04:16.009596 http.c:660              => Send header:
21:04:16.064701 http.c:648              <= Recv header, 0000000044 bytes (0x0000002c)
21:04:16.064722 http.c:660              <= Recv header: HTTP/1.1 407 Proxy Authentication Required
21:04:16.064755 http.c:648              <= Recv header, 0000000027 bytes (0x0000001b)
21:04:16.064758 http.c:660              <= Recv header: Proxy-Authenticate: Basic
21:04:16.064784 http.c:648              <= Recv header, 0000000019 bytes (0x00000013)
21:04:16.064786 http.c:660              <= Recv header: Content-Length: 0
21:04:16.064796 http.c:648              <= Recv header, 0000000002 bytes (0x00000002)
21:04:16.064798 http.c:660              <= Recv header:
21:04:16.064805 http.c:701              == Info: Establish HTTP proxy tunnel to <hidden host here>:443
21:04:16.064814 http.c:701              == Info: Proxy auth using Basic with user 'abc'
21:04:16.064869 http.c:648              => Send header, 0000000178 bytes (0x000000b2)
21:04:16.064877 http.c:660              => Send header: CONNECT <hidden host here>:443 HTTP/1.1
21:04:16.064879 http.c:660              => Send header: Host: <hidden host here>:443
21:04:16.064882 http.c:660              => Send header: Proxy-Authorization: Basic <redacted>
21:04:16.064884 http.c:660              => Send header: User-Agent: git/2.39.1
21:04:16.064886 http.c:660              => Send header: Proxy-Connection: Keep-Alive
21:04:16.064888 http.c:660              => Send header:
21:04:16.154312 http.c:648              <= Recv header, 0000000017 bytes (0x00000011)
21:04:16.154355 http.c:660              <= Recv header: HTTP/1.1 200 OK
21:04:16.154444 http.c:648              <= Recv header, 0000000019 bytes (0x00000013)
21:04:16.154450 http.c:660              <= Recv header: Content-Length: 0
21:04:16.154459 http.c:701              == Info: Ignoring Content-Length in CONNECT 200 response
21:04:16.154468 http.c:648              <= Recv header, 0000000002 bytes (0x00000002)
21:04:16.154472 http.c:660              <= Recv header:
21:04:16.154476 http.c:701              == Info: CONNECT phase completed
21:04:16.154485 http.c:701              == Info: CONNECT tunnel established, response 200
21:04:16.154647 http.c:701              == Info: ALPN: offers h2
21:04:16.154696 http.c:701              == Info: ALPN: offers http/1.1
21:04:16.162334 http.c:701              == Info:  CAfile: /etc/ssl/cert.pem
21:04:16.162350 http.c:701              == Info:  CApath: none
21:04:16.162622 http.c:701              == Info: [CONN-0-0][CF-SSL] (304) (OUT), TLS handshake, Client hello (1):
21:04:16.214366 http.c:701              == Info: [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Server hello (2):
21:04:16.214584 http.c:701              == Info: [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Unknown (8):
21:04:16.214632 http.c:701              == Info: [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Certificate (11):
21:04:16.251830 http.c:701              == Info: [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, CERT verify (15):
21:04:16.252072 http.c:701              == Info: [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Finished (20):
21:04:16.252138 http.c:701              == Info: [CONN-0-0][CF-SSL] (304) (OUT), TLS handshake, Finished (20):
21:04:16.252150 http.c:701              == Info: SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
21:04:16.252153 http.c:701              == Info: ALPN: server accepted http/1.1
21:04:16.252159 http.c:701              == Info: Server certificate:
21:04:16.252168 http.c:701              == Info:  subject: O=gomitmproxy; CN=<hidden host here>
21:04:16.252173 http.c:701              == Info:  start date: May 24 13:04:16 2023 GMT
21:04:16.252177 http.c:701              == Info:  expire date: Jun  7 13:04:16 2023 GMT
21:04:16.252187 http.c:701              == Info:  subjectAltName: host "<hidden host here>" matched cert's "<hidden host here>"
21:04:16.252192 http.c:701              == Info:  issuer: O=kube-ca
21:04:16.252195 http.c:701              == Info:  SSL certificate verify ok.
21:04:16.252230 http.c:648              => Send header, 0000000243 bytes (0x000000f3)
21:04:16.252236 http.c:660              => Send header: GET /root/e2eproject/info/refs?service=git-upload-pack HTTP/1.1
21:04:16.252239 http.c:660              => Send header: Host: <hidden host here>
21:04:16.252241 http.c:660              => Send header: User-Agent: git/2.39.1
21:04:16.252244 http.c:660              => Send header: Accept: */*
21:04:16.252246 http.c:660              => Send header: Accept-Encoding: deflate, gzip
21:04:16.252248 http.c:660              => Send header: Accept-Language: zh-CN, *;q=0.9
21:04:16.252250 http.c:660              => Send header: Pragma: no-cache
21:04:16.252252 http.c:660              => Send header: Git-Protocol: version=2
21:04:16.252254 http.c:660              => Send header:
21:04:16.670472 http.c:701              == Info: Mark bundle as not supporting multiuse
21:04:16.670496 http.c:648              <= Recv header, 0000000032 bytes (0x00000020)
21:04:16.670500 http.c:660              <= Recv header: HTTP/1.1 301 Moved Permanently
21:04:16.670503 http.c:648              <= Recv header, 0000000021 bytes (0x00000015)
21:04:16.670504 http.c:660              <= Recv header: Content-Length: 151
21:04:16.670507 http.c:648              <= Recv header, 0000000025 bytes (0x00000019)
21:04:16.670508 http.c:660              <= Recv header: Cache-Control: no-cache
21:04:16.670510 http.c:648              <= Recv header, 0000000025 bytes (0x00000019)
21:04:16.670512 http.c:660              <= Recv header: Content-Type: text/html
21:04:16.670513 http.c:648              <= Recv header, 0000000037 bytes (0x00000025)
21:04:16.670514 http.c:660              <= Recv header: Date: Wed, 31 May 2023 13:04:16 GMT
21:04:16.670517 http.c:648              <= Recv header, 0000000097 bytes (0x00000061)
21:04:16.670589 http.c:660              <= Recv header: Location: https://<hidden host here>/root/e2eproject.git/info/refs?service=git-upload-pack
21:04:16.670592 http.c:648              <= Recv header, 0000000064 bytes (0x00000040)
21:04:16.670594 http.c:660              <= Recv header: Strict-Transport-Security: max-age=31536000; includeSubDomains
21:04:16.670595 http.c:648              <= Recv header, 0000000010 bytes (0x0000000a)
21:04:16.670596 http.c:660              <= Recv header: Via: alb
21:04:16.670598 http.c:648              <= Recv header, 0000000042 bytes (0x0000002a)
21:04:16.670599 http.c:660              <= Recv header: X-Request-Id: 01H1RYHDTXEZ8DQCYRG4Q2BFB3
21:04:16.670600 http.c:648              <= Recv header, 0000000021 bytes (0x00000015)
21:04:16.670601 http.c:660              <= Recv header: X-Runtime: 0.013325
21:04:16.670604 http.c:648              <= Recv header, 0000000002 bytes (0x00000002)
21:04:16.670605 http.c:660              <= Recv header:
21:04:16.670608 http.c:701              == Info: Ignoring the response-body
21:04:16.670660 http.c:701              == Info: Connection #0 to host 127.0.0.1 left intact
21:04:16.670674 http.c:701              == Info: Issue another request to this URL: 'https://<hidden host here>/root/e2eproject.git/info/refs?service=git-upload-pack'
21:04:16.670753 http.c:701              == Info: Couldn't find host <hidden host here> in the (nil) file; using defaults
21:04:16.670764 http.c:701              == Info: Found bundle for host: 0x6000009a4390 [serially]
21:04:16.670767 http.c:701              == Info: Can not multiplex, even if we wanted to
21:04:16.670782 http.c:701              == Info: Re-using existing connection #0 with proxy 127.0.0.1
21:04:16.670884 http.c:648              => Send header, 0000000247 bytes (0x000000f7)
21:04:16.670898 http.c:660              => Send header: GET /root/e2eproject.git/info/refs?service=git-upload-pack HTTP/1.1
21:04:16.670901 http.c:660              => Send header: Host: <hidden host here>
21:04:16.670903 http.c:660              => Send header: User-Agent: git/2.39.1
21:04:16.670905 http.c:660              => Send header: Accept: */*
21:04:16.670907 http.c:660              => Send header: Accept-Encoding: deflate, gzip
21:04:16.670909 http.c:660              => Send header: Accept-Language: zh-CN, *;q=0.9
21:04:16.670911 http.c:660              => Send header: Pragma: no-cache
21:04:16.670913 http.c:660              => Send header: Git-Protocol: version=2
21:04:16.670915 http.c:660              => Send header:
21:04:16.833114 http.c:701              == Info: Mark bundle as not supporting multiuse
21:04:16.833175 http.c:648              <= Recv header, 0000000027 bytes (0x0000001b)
21:04:16.833188 http.c:660              <= Recv header: HTTP/1.1 401 Unauthorized
21:04:16.833195 http.c:648              <= Recv header, 0000000020 bytes (0x00000014)
21:04:16.833198 http.c:660              <= Recv header: Content-Length: 26
21:04:16.833204 http.c:648              <= Recv header, 0000000025 bytes (0x00000019)
21:04:16.833207 http.c:660              <= Recv header: Cache-Control: no-cache
21:04:16.833212 http.c:648              <= Recv header, 0000000041 bytes (0x00000029)
21:04:16.833225 http.c:660              <= Recv header: Content-Type: text/plain; charset=utf-8
21:04:16.833229 http.c:648              <= Recv header, 0000000037 bytes (0x00000025)
21:04:16.833232 http.c:660              <= Recv header: Date: Wed, 31 May 2023 13:04:16 GMT
21:04:16.833236 http.c:648              <= Recv header, 0000000050 bytes (0x00000032)
21:04:16.833953 http.c:660              <= Recv header: Referrer-Policy: strict-origin-when-cross-origin
21:04:16.833963 http.c:648              <= Recv header, 0000000064 bytes (0x00000040)
21:04:16.833967 http.c:660              <= Recv header: Strict-Transport-Security: max-age=31536000; includeSubDomains
21:04:16.833972 http.c:648              <= Recv header, 0000000010 bytes (0x0000000a)
21:04:16.833975 http.c:660              <= Recv header: Via: alb
21:04:16.833980 http.c:648              <= Recv header, 0000000040 bytes (0x00000028)
21:04:16.833984 http.c:660              <= Recv header: Www-Authenticate: Basic realm="GitLab"
21:04:16.833987 http.c:648              <= Recv header, 0000000033 bytes (0x00000021)
21:04:16.833990 http.c:660              <= Recv header: X-Content-Type-Options: nosniff
21:04:16.833993 http.c:648              <= Recv header, 0000000028 bytes (0x0000001c)
21:04:16.833996 http.c:660              <= Recv header: X-Download-Options: noopen
21:04:16.833999 http.c:648              <= Recv header, 0000000029 bytes (0x0000001d)
21:04:16.834253 http.c:660              <= Recv header: X-Frame-Options: SAMEORIGIN
21:04:16.834259 http.c:648              <= Recv header, 0000000041 bytes (0x00000029)
21:04:16.834262 http.c:660              <= Recv header: X-Permitted-Cross-Domain-Policies: none
21:04:16.834265 http.c:648              <= Recv header, 0000000042 bytes (0x0000002a)
21:04:16.834268 http.c:660              <= Recv header: X-Request-Id: 01H1RYHDYQWQVTGHXK56AT0SNB
21:04:16.834271 http.c:648              <= Recv header, 0000000021 bytes (0x00000015)
21:04:16.834274 http.c:660              <= Recv header: X-Runtime: 0.041236
21:04:16.834277 http.c:648              <= Recv header, 0000000033 bytes (0x00000021)
21:04:16.834280 http.c:660              <= Recv header: X-Xss-Protection: 1; mode=block
21:04:16.834285 http.c:648              <= Recv header, 0000000002 bytes (0x00000002)
21:04:16.834288 http.c:660              <= Recv header:
21:04:16.834323 http.c:701              == Info: Connection #0 to host 127.0.0.1 left intact
21:04:16.835377 run-command.c:655       trace: run_command: 'git credential-osxkeychain get'
21:04:16.859388 git.c:750               trace: exec: git-credential-osxkeychain get
21:04:16.860101 run-command.c:655       trace: run_command: git-credential-osxkeychain get

proxy server log

2023/05/31 21:04:12 96670#34 [info] start listening to [::]:8080
2023/05/31 21:04:15 96670#34 [debug] id=100001: accepted connection from 127.0.0.1:64057
2023/05/31 21:04:15 96670#18 [debug] id=100001: waiting for request
2023/05/31 21:04:16 96670#18 [debug] id=100001-1: handle request CONNECT http://<hidden host here>:443
2023/05/31 21:04:16 96670#18 [debug] id=100001-1: proxy auth required
2023/05/31 21:04:16 96670#18 [debug] id=100001: waiting for request
2023/05/31 21:04:16 96670#18 [debug] id=100001-2: handle request CONNECT http://<hidden host here>:443
2023/05/31 21:04:16 96670#18 [debug] id=100001-2: connecting to host: <hidden host here>:443
2023/05/31 21:04:16 96670#18 [debug] id=100001-2: connecting to tcp://<hidden host here>:443
2023/05/31 21:04:16 96670#18 [debug] id=100001-2: attempting MITM for connection
2023/05/31 21:04:16 96670#18 [debug] mitm: cache miss for <hidden host here>
2023/05/31 21:04:16 96670#18 [debug] id=100001-2-1: waiting for request
2023/05/31 21:04:16 96670#18 [debug] id=100001-2-1-1: handle request GET https://<hidden host here>/root/e2eproject/info/refs?service=git-upload-pack
2023/05/31 21:04:16 96670#18 [debug] id=100001-2-1-1: received response 301 Moved Permanently
2023/05/31 21:04:16 96670#18 [debug] id=100001-2-1: waiting for request
2023/05/31 21:04:16 96670#18 [debug] id=100001-2-1-2: handle request GET https://<hidden host here>/root/e2eproject.git/info/refs?service=git-upload-pack
2023/05/31 21:04:16 96670#18 [debug] id=100001-2-1-2: received response 401 Unauthorized
2023/05/31 21:04:16 96670#18 [debug] id=100001-2-1: waiting for request
ameshkov commented 1 year ago

@airycanon hi, thank you for reporting this!

It seems the only change required for the fix is this: https://github.com/AdguardTeam/gomitmproxy/commit/147e33e57cbace9ea6b33a02b9479e010bcb288a#diff-fd4820e2c415e0057b0e67d294303bdde3b98c2731b6dbd1e2b0f79349744499R276

Other changes in the PR are not necessary.

airycanon commented 1 year ago

@ameshkov Thanks for your response. I need to implement the validation of the Proxy-Authorization within the OnRequest function, so I added a method to retrieve the Parent from the Context. Additionally, I also needed to ensure that the connection remains open, so I included a check for the StatusCode in the isClosing function.