flomesh-io / pipy

Pipy is a programmable proxy for the cloud, edge and IoT.
https://flomesh.io/pipy
Other
743 stars 70 forks source link

pipy crash when visit admin-port #142

Closed ethinx closed 1 year ago

ethinx commented 1 year ago

How to reproduce

Start pipy, it may crash when visit http://localhost:6060/metrics, I used wrk

wrk -c 10 -t 4 -d 300s http://localhost:6060/metrics

I'm not sure but try dropping OS cache before testing may be a bit easier to reproduce.

sync
echo 3 | sudo tee /proc/sys/vm/drop_caches
$ pipy --threads=4 --admin-port=6060 -e 'pipy().listen(8000).serveHTTP(()=>new Message("hello\n"))'
2023-01-04 15:33:32.598 [INF] [config]
2023-01-04 15:33:32.598 [INF] [config] Module
2023-01-04 15:33:32.598 [INF] [config] =======
2023-01-04 15:33:32.598 [INF] [config]
2023-01-04 15:33:32.598 [INF] [config]  [Listen on 8000 at 0.0.0.0]
2023-01-04 15:33:32.598 [INF] [config]  ----->|
2023-01-04 15:33:32.598 [INF] [config]        |
2023-01-04 15:33:32.598 [INF] [config]       serveHTTP -->|
2023-01-04 15:33:32.598 [INF] [config]                    |
2023-01-04 15:33:32.598 [INF] [config]  <-----------------|
2023-01-04 15:33:32.598 [INF] [config]
2023-01-04 15:33:32.599 [INF] [listener] Listening on TCP port 8000 at 0.0.0.0
2023-01-04 15:33:32.599 [INF] [admin] Starting admin service...
2023-01-04 15:33:32.600 [INF] [listener] Listening on TCP port 6060 at ::
2023-01-04 15:33:32.600 [INF] Running 4 worker threads...
Segmentation fault (core dumped)

btw, start pipy again soon after it crashed, it may complain the admin-port is being used. However I can't find anything about port 6060 via lsof or ss command. This may be related to the OS but the crash issue is the point.

$ pipy --threads=4 --admin-port=6060 -e 'pipy().listen(8000).serveHTTP(()=>new Message("hello\n"))'
2023-01-04 15:44:13.474 [INF] [config]
2023-01-04 15:44:13.474 [INF] [config] Module
2023-01-04 15:44:13.474 [INF] [config] =======
2023-01-04 15:44:13.474 [INF] [config]
2023-01-04 15:44:13.475 [INF] [config]  [Listen on 8000 at 0.0.0.0]
2023-01-04 15:44:13.475 [INF] [config]  ----->|
2023-01-04 15:44:13.475 [INF] [config]        |
2023-01-04 15:44:13.475 [INF] [config]       serveHTTP -->|
2023-01-04 15:44:13.475 [INF] [config]                    |
2023-01-04 15:44:13.475 [INF] [config]  <-----------------|
2023-01-04 15:44:13.475 [INF] [config]
2023-01-04 15:44:13.475 [INF] [listener] Listening on TCP port 8000 at 0.0.0.0
2023-01-04 15:44:13.475 [INF] [admin] Starting admin service...
[listener] Cannot start listening on TCP port 6060 at ::: bind: Address already in use

Expect behavior

Everything is fine and no crash

Version info

OS:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04 LTS
Release:        22.04
Codename:       jammy

$ uname -r
5.15.0-39-generic

pipy:

$ pipy -v
Version     : nightly-202301041410
Commit      : 55c9e5d546f1e415688decfee7d823983577dd4c
Commit Date : Wed, 4 Jan 2023 11:37:10 +0800
Host        : Linux-5.15.0-39-generic x86_64
OpenSSL     : OpenSSL 1.1.1q  5 Jul 2022
Builtin GUI : No
Samples     : No
ethinx commented 1 year ago

follow-up: this problem can't be reproduced after recent updates.