klzgrad / naiveproxy

Make a fortune quietly
BSD 3-Clause "New" or "Revised" License
6.31k stars 868 forks source link

caddy 的内存占用异常高 #610

Closed zwyyy456 closed 4 months ago

zwyyy456 commented 4 months ago

caddy 一共使用了两个插件,一个是 caddy-trojan ,另一个是 naive-proxy 作者修改的 forwardproxy 插件,几个 vps ,凡是按照上面的配置文件部署了 caddy 的,内存占用都是起码 500-600M ,我 24g 的甲骨文 arm 机器,caddy 内存占用甚至达到了 5-6g ,问了下 chatgpt ,也没给出什么实质性的建议;

重启 caddy 之后,内存占用倒是降下来了,但是似乎随着时间增长,占用会逐渐升高,请问要如何排查问题呢?Caddyfile 的内容如下,对用户名、密码、域名等敏感信息作了修改;


{
    order trojan before file_server
    servers :4443 {
        listener_wrappers {
            trojan
        }
    }
    log {
        output file /var/log/caddy/default.log {
            roll_size 10MiB
        }
    }
    trojan {
        caddy
        no_proxy
        users user tro_password
    }
}
tro.arm-br.example.com {
    reverse_proxy https://tro-arm-br.example.com:4443
}
naive.arm-br.example.com {
    reverse_proxy https://arm-br.example.com:443 
}
plex.arm-br.example.com {
    reverse_proxy arm-br.example.com:32400
}
qbit.arm-br.example.com {
    reverse_proxy arm-br.example.com:28080
}
cd2.arm-br.example.com {
    reverse_proxy arm-br.example.com:19798
}
status.arm-br.example.com {
    reverse_proxy arm-br.example.com:10182
}

:443, arm-br.example.com {
    tls user456@gmail.com
    route {
        forward_proxy {
            basic_auth user naive_password
            hide_ip
            hide_via
            probe_resistance
        }
        file_server {
            root /usr/share/caddy
        }
    }
}
:4443, tro-arm-br.example.com {
tls user456@gmail.com 
    route {
        trojan {
            connect_method
            websocket
        }
        file_server {
            root /usr/share/caddy
        }
    }
}

:6443, arm-brv6.example.com {
    tls user456@gmail.com
    route {
        forward_proxy {
            basic_auth user naive_password
            hide_ip
            hide_via
            probe_resistance
        }
        file_server {
            root /usr/share/caddy
        }
    }
}

这里贴出一部分日志

{"level":"info","ts":1704763488.7812674,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1704763488.7820559,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704763488.7821221,"logger":"http","msg":"enabling HTTP/3 listener","addr":":4443"}
{"level":"info","ts":1704763488.7824142,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704763488.7824407,"logger":"http","msg":"enabling HTTP/3 listener","addr":":6443"}
{"level":"info","ts":1704763488.7826765,"logger":"http.log","msg":"server running","name":"srv2","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704763488.7827032,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704763488.7827075,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["naive.lw.ggll.eu.org","plex.lw.ggll.eu.org","qbit.lw.ggll.eu.org","tro.lw.ggll.eu.org","cd2.lw.ggll.eu.org","tro-lw.ggll.eu.org","lwv6.ggll.eu.org","status.lw.ggll.eu.org"]}
{"level":"warn","ts":1704763488.8449705,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/var/lib/caddy/.local/share/caddy","instance":"f6b0280a-020c-47f0-afa4-168ce77cd684","try_again":1704849888.8449664,"try_again_in":86399.999999489}
{"level":"info","ts":1704763488.8450851,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1704763488.8468266,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
{"level":"info","ts":1704763488.8481233,"msg":"serving initial configuration"}
{"level":"error","ts":1704763522.5706208,"logger":"caddy.listeners.trojan","msg":"read prefix error: read tcp 112.15.179.204:55125 -> 38.72.148.44:4443: read: EOF"}
{"level":"error","ts":1704768338.0250442,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for '38.72.148.44'"}
{"level":"error","ts":1704771357.2400272,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: tls: first record does not look like a TLS handshake"}
{"level":"error","ts":1704772503.677092,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: read tcp 38.72.148.44:4443->167.248.133.127:43178: read: connection reset by peer"}
{"level":"error","ts":1704772503.7557719,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for '38.72.148.44'"}
{"level":"error","ts":1704772506.9289355,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: tls: first record does not look like a TLS handshake"}
{"level":"error","ts":1704772510.378716,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: tls: first record does not look like a TLS handshake"}
{"level":"error","ts":1704772510.4845634,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: tls: first record does not look like a TLS handshake"}
{"level":"error","ts":1704775922.9165707,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for '38.72.148.44'"}
{"level":"error","ts":1704779861.6374896,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for 'naive-lw.ggll.eu.org'"}
{"level":"error","ts":1704779861.9872792,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for 'naive-lw.ggll.eu.org'"}
{"level":"error","ts":1704784934.799494,"logger":"http.log.error","msg":"dial tcp 38.72.148.44:10182: connect: connection refused","request":{"remote_ip":"104.237.134.123","remote_port":"43238","client_ip":"104.237.134.123","proto":"HTTP/1.1","method":"GET","host":"status.lw.ggll.eu.org","uri":"/","headers":{"Accept-Encoding":["gzip"],"Connection":["close"],"User-Agent":["Go-http-client/1.1"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"","server_name":"status.lw.ggll.eu.org"}},"duration":0.00567612,"status":502,"err_id":"evhj4586b","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
{"level":"error","ts":1704785624.133557,"logger":"http.log.error","msg":"dial tcp 38.72.148.44:28080: connect: connection refused","request":{"remote_ip":"139.177.207.147","remote_port":"58692","client_ip":"139.177.207.147","proto":"HTTP/1.1","method":"GET","host":"qbit.lw.ggll.eu.org","uri":"/","headers":{"Accept-Encoding":["gzip"],"Connection":["close"],"User-Agent":["Go-http-client/1.1"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"","server_name":"qbit.lw.ggll.eu.org"}},"duration":0.276857775,"status":502,"err_id":"zr76ydb3z","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
{"level":"info","ts":1704789498.6327744,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
{"level":"warn","ts":1704789498.6343508,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
{"level":"info","ts":1704789498.6344306,"logger":"http","msg":"servers shutting down with eternal grace period"}
{"level":"info","ts":1704789498.6832166,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1704789498.6832995,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
{"level":"info","ts":1704789499.0957417,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1704789499.0968235,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1704789499.096848,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1704789499.0968544,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv1"}
{"level":"info","ts":1704789499.0968592,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv2"}
{"level":"info","ts":1704789499.0977018,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704789499.097743,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1704789499.0981522,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704789499.098187,"logger":"http","msg":"enabling HTTP/3 listener","addr":":4443"}
{"level":"info","ts":1704789499.0985396,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704789499.0985725,"logger":"http","msg":"enabling HTTP/3 listener","addr":":6443"}
{"level":"info","ts":1704789499.0986316,"logger":"http.log","msg":"server running","name":"srv2","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704789499.0986369,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["tro-lw.ggll.eu.org","lwv6.ggll.eu.org","status.lw.ggll.eu.org","naive.lw.ggll.eu.org","plex.lw.ggll.eu.org","qbit.lw.ggll.eu.org","tro.lw.ggll.eu.org","cd2.lw.ggll.eu.org"]}
{"level":"info","ts":1704789499.1366014,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0004d9b80"}
{"level":"info","ts":1704789499.1503842,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
{"level":"info","ts":1704789499.1612566,"msg":"serving initial configuration"}
{"level":"info","ts":1704789499.1748154,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/var/lib/caddy/.local/share/caddy"}
{"level":"info","ts":1704789499.176846,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"error","ts":1704790385.6787434,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","upstream":"lw.ggll.eu.org:19798","duration":0.000933625,"request":{"remote_ip":"112.15.179.204","remote_port":"4395","client_ip":"112.15.179.204","proto":"HTTP/3.0","method":"GET","host":"cd2.lw.ggll.eu.org","uri":"/_framework/icudt.dat","headers":{"X-Forwarded-For":["112.15.179.204"],"X-Forwarded-Proto":["https"],"Sec-Fetch-Mode":["cors"],"Sec-Fetch-Dest":["empty"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36 Edg/112.0.1722.54"],"Cache-Control":["max-age=0"],"Dnt":["1"],"Accept-Language":["zh-CN,zh;q=0.9,en;q=0.8,en-US;q=0.7"],"Sec-Fetch-Site":["same-origin"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://cd2.lw.ggll.eu.org/service-worker.js"],"X-Forwarded-Host":["cd2.lw.ggll.eu.org"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"cd2.lw.ggll.eu.org"}},"error":"writing: H3_REQUEST_CANCELLED"}
{"level":"error","ts":1704790385.6794074,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","upstream":"lw.ggll.eu.org:19798","duration":0.000900173,"request":{"remote_ip":"112.15.179.204","remote_port":"4395","client_ip":"112.15.179.204","proto":"HTTP/3.0","method":"GET","host":"cd2.lw.ggll.eu.org","uri":"/_framework/icudt_no_CJK.dat","headers":{"Dnt":["1"],"Accept-Language":["zh-CN,zh;q=0.9,en;q=0.8,en-US;q=0.7"],"Cache-Control":["max-age=0"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36 Edg/112.0.1722.54"],"Accept":["*/*"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["cd2.lw.ggll.eu.org"],"Sec-Fetch-Dest":["empty"],"X-Forwarded-For":["112.15.179.204"],"Referer":["https://cd2.lw.ggll.eu.org/service-worker.js"],"Sec-Fetch-Mode":["cors"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Site":["same-origin"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"cd2.lw.ggll.eu.org"}},"error":"writing: H3_REQUEST_CANCELLED"}
{"level":"error","ts":1704790488.9171484,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for '38.72.148.44:4443'"}

本人不太懂 go 与网络的相关知识,还请作者有空的时候能帮忙看看,或者给出一点排查问题的建议。

zwyyy456 commented 4 months ago

假设被转发给的端口,例如 10182,在该服务器上并没有应用监听,会有什么负面效果吗?

klzgrad commented 4 months ago

内存泄漏,并不容易定位问题,建议你设置定时重启