moooofly
commented
5 years ago 测试
本地启动代理
[#47#root@ubuntu-1604 /go/src/github.com/elazarl/goproxy/examples/goproxy-eavesdropper]$./goproxy-eavesdropper -v默认监听 8080 端口
[#29#root@ubuntu-1604 /go/src/github.com/elazarl/goproxy]$netstat -nltp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
...
tcp6 0 0 :::8080 :::* LISTEN 12535/goproxy-eaves
...测试针对 https://example.com/ 的代理
基于 curl 访问 proxy ，未指定 -k 时
[#43#root@ubuntu-1604 ~]$curl -Lv --proxy-header "Custom-Info: moooofly" --proxy http://localhost:8080 https://example.com/
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Establish HTTP proxy tunnel to example.com:443
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
> Custom-Info: moooofly
>
< HTTP/1.0 200 OK
<
* Proxy replied OK to CONNECT request
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
* Closing connection 0
curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
[#44#root@ubuntu-1604 ~]$由于证书原因，proxy 主动断开了 tcp 连接
[#47#root@ubuntu-1604 /go/src/github.com/elazarl/goproxy/examples/goproxy-eavesdropper]$./goproxy-eavesdropper -v
2019/01/22 15:47:56 [016] INFO: Running 3 CONNECT handlers
2019/01/22 15:47:56 [016] INFO: on 1th handler: &{2 <nil> 0x65d290} example.com:443
2019/01/22 15:47:56 [016] INFO: Assuming CONNECT is TLS, mitm proxying it
2019/01/22 15:47:56 [016] INFO: signing for example.com
2019/01/22 15:47:56 [016] INFO: Exiting on EOF抓包信息
基于 curl 访问 proxy ，指定 -k 时
[#44#root@ubuntu-1604 ~]$curl -Lv --proxy-header "Custom-Info: moooofly" --proxy http://localhost:8080 https://example.com/ -k
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Establish HTTP proxy tunnel to example.com:443
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
> Custom-Info: moooofly
>
< HTTP/1.0 200 OK
<
* Proxy replied OK to CONNECT request
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification SKIPPED
* server certificate status verification SKIPPED
* common name: example.com (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: O=GoProxy untrusted MITM proxy Inc,CN=example.com
* start date: Thu, 01 Jan 1970 00:00:00 GMT
* expire date: Fri, 31 Dec 2049 00:00:00 GMT
* issuer: C=IL,ST=Center,L=Lod,O=GoProxy,OU=GoProxy,CN=goproxy.github.io,EMAIL=elazarl@gmail.com
* compression: NULL
* ALPN, server did not agree to a protocol
> GET / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: max-age=604800
< Connection: close
< Content-Type: text/html; charset=UTF-8
< Date: Tue, 22 Jan 2019 07:48:07 GMT
< Etag: "1541025663"
< Expires: Tue, 29 Jan 2019 07:48:07 GMT
< Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
< Server: ECS (sjc/4E8B)
< Transfer-Encoding: chunked
< Vary: Accept-Encoding
< X-Cache: HIT
<
<!doctype html>
<html>
<head>
<title>Example Domain</title>
<meta charset="utf-8" />
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<style type="text/css">
body {
background-color: #f0f0f2;
margin: 0;
padding: 0;
font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
}
div {
width: 600px;
margin: 5em auto;
padding: 50px;
background-color: #fff;
border-radius: 1em;
}
a:link, a:visited {
color: #38488f;
text-decoration: none;
}
@media (max-width: 700px) {
body {
background-color: #fff;
}
div {
width: auto;
margin: 0 auto;
border-radius: 0;
padding: 1em;
}
}
</style>
</head>
<body>
<div>
<h1>Example Domain</h1>
<p>This domain is established to be used for illustrative examples in documents. You may use this
domain in examples without prior coordination or asking for permission.</p>
<p><a href="http://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>
* Closing connection 0
[#45#root@ubuntu-1604 ~]$调整后，成功输出
2019/01/22 15:48:06 [017] INFO: Running 3 CONNECT handlers
2019/01/22 15:48:06 [017] INFO: on 1th handler: &{2 <nil> 0x65d290} example.com:443
2019/01/22 15:48:06 [017] INFO: Assuming CONNECT is TLS, mitm proxying it
2019/01/22 15:48:06 [017] INFO: signing for example.com
2019/01/22 15:48:06 [018] INFO: req example.com:443
2019/01/22 15:48:06 [018] INFO: Sending request GET https://example.com:443/
2019/01/22 15:48:07 [018] INFO: resp 200 OK
2019/01/22 15:48:07 [017] INFO: Exiting on EOF抓包信息
测试针对 https://apineo.llsapp.com/ 的代理
基于 curl 访问 proxy ，未指定 -k 时
2019/01/22 16:33:36 [006] INFO: Running 3 CONNECT handlers
2019/01/22 16:33:36 [006] INFO: on 1th handler: &{2 <nil> 0x65d290} apineo.llsapp.com:443
2019/01/22 16:33:36 [006] INFO: Assuming CONNECT is TLS, mitm proxying it
2019/01/22 16:33:36 [006] INFO: signing for apineo.llsapp.com
2019/01/22 16:33:36 [006] INFO: Exiting on EOFcurl 失败日志（证书问题）
[#49#root@ubuntu-1604 ~]$curl -Lv --proxy-header "Custom-Info: moooofly" --proxy http://localhost:8080 https://apineo.llsapp.com
* Rebuilt URL to: https://apineo.llsapp.com/
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Establish HTTP proxy tunnel to apineo.llsapp.com:443
> CONNECT apineo.llsapp.com:443 HTTP/1.1
> Host: apineo.llsapp.com:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
> Custom-Info: moooofly
>
< HTTP/1.0 200 OK
<
* Proxy replied OK to CONNECT request
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
* Closing connection 0
curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
[#50#root@ubuntu-1604 ~]$
基于 curl 访问 proxy ，指定 -k 时
2019/01/22 16:33:51 [007] INFO: Running 3 CONNECT handlers
2019/01/22 16:33:51 [007] INFO: on 1th handler: &{2 <nil> 0x65d290} apineo.llsapp.com:443
2019/01/22 16:33:51 [007] INFO: Assuming CONNECT is TLS, mitm proxying it
2019/01/22 16:33:51 [007] INFO: signing for apineo.llsapp.com
2019/01/22 16:33:51 [008] INFO: req apineo.llsapp.com:443
2019/01/22 16:33:51 [008] INFO: Sending request GET https://apineo.llsapp.com:443/
2019/01/22 16:33:51 [008] INFO: resp 301 Moved Permanently
2019/01/22 16:33:51 [007] INFO: Exiting on EOF
2019/01/22 16:33:51 [009] INFO: Running 3 CONNECT handlers
2019/01/22 16:33:51 [009] INFO: on 1th handler: &{2 <nil> 0x65d290} www.liulishuo.com:443
2019/01/22 16:33:51 [009] INFO: Assuming CONNECT is TLS, mitm proxying it
2019/01/22 16:33:51 [009] INFO: signing for www.liulishuo.com
2019/01/22 16:33:51 [010] INFO: req www.liulishuo.com:443
2019/01/22 16:33:51 [010] INFO: Sending request GET https://www.liulishuo.com:443/
2019/01/22 16:33:52 [010] INFO: resp 200 OK
2019/01/22 16:33:52 [009] INFO: Exiting on EOFcurl 成功日志
[#50#root@ubuntu-1604 ~]$curl -Lv --proxy-header "Custom-Info: moooofly" --proxy http://localhost:8080 https://apineo.llsapp.com -k
* Rebuilt URL to: https://apineo.llsapp.com/
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Establish HTTP proxy tunnel to apineo.llsapp.com:443
> CONNECT apineo.llsapp.com:443 HTTP/1.1
> Host: apineo.llsapp.com:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
> Custom-Info: moooofly
>
< HTTP/1.0 200 OK
<
* Proxy replied OK to CONNECT request
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification SKIPPED
* server certificate status verification SKIPPED
* common name: apineo.llsapp.com (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: O=GoProxy untrusted MITM proxy Inc,CN=apineo.llsapp.com
* start date: Thu, 01 Jan 1970 00:00:00 GMT
* expire date: Fri, 31 Dec 2049 00:00:00 GMT
* issuer: C=IL,ST=Center,L=Lod,O=GoProxy,OU=GoProxy,CN=goproxy.github.io,EMAIL=elazarl@gmail.com
* compression: NULL
* ALPN, server did not agree to a protocol
> GET / HTTP/1.1
> Host: apineo.llsapp.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Cache-Control: no-cache
< Connection: close
< Content-Language: en
< Content-Type: text/html
< Location: https://www.liulishuo.com
< Transfer-Encoding: chunked
< Vary: Origin
< X-Kong-Proxy-Latency: 0
< X-Kong-Upstream-Latency: 6
< X-Login: 0
< X-Request-Id: e884b66a-bf7a-42e2-94d3-75a6bde5b5b3
< X-Runtime: 0.004709
<
* Closing connection 0
* Issue another request to this URL: 'https://www.liulishuo.com'
* Rebuilt URL to: https://www.liulishuo.com/
* Hostname localhost was found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#1)
* Establish HTTP proxy tunnel to www.liulishuo.com:443
> CONNECT www.liulishuo.com:443 HTTP/1.1
> Host: www.liulishuo.com:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
> Custom-Info: moooofly
>
< HTTP/1.0 200 OK
<
* Proxy replied OK to CONNECT request
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification SKIPPED
* server certificate status verification SKIPPED
* common name: www.liulishuo.com (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: O=GoProxy untrusted MITM proxy Inc,CN=www.liulishuo.com
* start date: Thu, 01 Jan 1970 00:00:00 GMT
* expire date: Fri, 31 Dec 2049 00:00:00 GMT
* issuer: C=IL,ST=Center,L=Lod,O=GoProxy,OU=GoProxy,CN=goproxy.github.io,EMAIL=elazarl@gmail.com
* compression: NULL
* ALPN, server did not agree to a protocol
> GET / HTTP/1.1
> Host: www.liulishuo.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Connection: close
< Content-Type: text/html; charset=utf-8
< Date: Tue, 22 Jan 2019 08:33:51 GMT
< Faas-Match: try-file
< Faas-Version: v0.3.0-1-45e6981e
< Last-Modified: Tue, 22 Jan 2019 06:38:07 GMT
< Transfer-Encoding: chunked
< Vary: Accept-Encoding
< X-Kong-Proxy-Latency: 0
< X-Kong-Upstream-Latency: 2
<
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>英语流利说 - 轻松学英语，用人工智能老师</title>
<meta name="description" content="流利说（LAIX）是领先的人工智能驱动的教育科技公司。作为智能教育的倡行者，流利说拥有一支业内领先的人工智能团队，其自主研发的人工智能英语老师，基于深度学习技术，能够为每一位用户提供个性化、自适应的学习课程。">
<meta name="keywords" content="AI老师,人工智能老师,懂你英语,懂你英语官网,LAIX,流利说,英语流利说,英语,liulishuo,雅思流利说,英语口语,口语评分,口语打分,外教,口语培训,雅思口语,职场口语,旅游口语,口语教材,手机,手机口语,外教,人工智能,智能教育,在线教育,智能英语云课堂,乡村教师赋能计划">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<link rel="shortcut icon" href="https://cdn.llscdn.com/faas/frontend/liulishuo.com/favicon.ico?t=1548126066775" type="image/x-icon">
<link rel="canonical" href="https://www.liulishuo.com">
<script type="text/javascript">
if(/Android|iPhone|mobile/i.test(navigator.userAgent)) {
var host = 'https://m.liulishuo.com' + window.location.pathname + window.location.search;
location.replace(host);
}
</script>
<!--[if lte IE 8]>
<script type="text/javascript" src="https://cdn.llscdn.com/faas/frontend/liulishuo.com/javascripts/libs/html5shiv.min.js?t=1548126066775"></script>
<![endif]-->
<link rel="stylesheet" href="https://cdn.llscdn.com/faas/frontend/liulishuo.com/styles/common.css?t=1548126066775">
<link rel="stylesheet" href="https://cdn.llscdn.com/faas/frontend/liulishuo.com/styles/index.css?t=1548126066775">
<script type="text/javascript" src="https://cdn.llscdn.com/faas/frontend/liulishuo.com/javascripts/libs/jquery-1.12.4.min.js?t=1548126066775"></script>
<!--[if lte IE 9]>
<script type="text/javascript" src="https://cdn.llscdn.com/faas/frontend/liulishuo.com/javascripts/libs/jquery.xdomainrequest.min.js?t=1548126066775"></script>
<![endif]-->
<script type="text/javascript" src="https://cdn.llscdn.com/faas/frontend/liulishuo.com/javascripts/libs/ckplayer.js?t=1548126066775"></script>
<script type="text/javascript" src="https://cdn.llscdn.com/faas/frontend/liulishuo.com/javascripts/common.js?t=1548126066775"></script>
<script type="text/javascript">
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-38218737-1', 'auto');
ga('send', 'pageview');
</script>
<script>
(function(h,o,t,j,a,r){
h.hj=h.hj||function(){(h.hj.q=h.hj.q||[]).push(arguments)};
h._hjSettings={hjid:1083213,hjsv:6};
a=o.getElementsByTagName('head')[0];
r=o.createElement('script');r.async=1;
r.src=t+h._hjSettings.hjid+j+h._hjSettings.hjsv;
a.appendChild(r);
})(window,document,'https://static.hotjar.com/c/hotjar-','.js?sv=');
</script>
</head>
<body>
...
</body>
</html>
* Closing connection 1
[#51#root@ubuntu-1604 ~]$
没做抓到针对 www.liulishuo.com. 的 DNS 解析（应该和缓存有关系）
相关 task
Package goproxy provides a customizable HTTP proxy library for Go (golang)
It supports regular HTTP proxy, HTTPS through CONNECT, and "hijacking" HTTPS connection using "Man in the Middle" style attack.
The intent of the proxy is to be usable with reasonable amount of traffic, yet customizable and programmable.
The proxy itself is simply a
net/httphandler.In order to use goproxy, one should set their browser to use goproxy as an HTTP proxy.
安装
帮助
https://github.com/elazarl/goproxy/blob/master/doc.go