search

ArroyoSystems / arroyo

Distributed stream processing engine in Rust
https://arroyo.dev
Apache License 2.0
3.8k stars 220 forks source link

Got error when call `/debug/pprof/heap` #764

Closed zhuliquan closed 3 weeks ago

zhuliquan commented 1 month ago

When I call admin server heap profile api, I got below error, and not heap.pb.gz file return

curl http://127.0.0.1:5114/debug/pprof/heap
curl: (52) Empty reply from server
2024-10-23 11:14:26.826726 [ERROR] <lib.rs:73> msg: arroyo_server_common: panicked at /home/zhuliquan/.cargo/registry/src/rsproxy.cn-8f6827c7555bfaf8/jemalloc_pprof-0.4.2/src/lib.rs:96:70:
called `Result::unwrap()` on an `Err` value: `name` or `mib` specifies an unknown/invalid value. panic.file="/home/zhuliquan/.cargo/registry/src/rsproxy.cn-8f6827c7555bfaf8/jemalloc_pprof-0.4.2/src/lib.rs" panic.line=96 panic.column=70
2024-10-23 11:14:29.125961 [ERROR] <lib.rs:73> msg: arroyo_server_common: panicked at /home/zhuliquan/.cargo/registry/src/rsproxy.cn-8f6827c7555bfaf8/once_cell-1.19.0/src/lib.rs:1311:25:
Lazy instance has previously been poisoned panic.file="/home/zhuliquan/.cargo/registry/src/rsproxy.cn-8f6827c7555bfaf8/once_cell-1.19.0/src/lib.rs" panic.line=1311 panic.column=25
2024-10-23 11:14:38.227303 [ERROR] <lib.rs:73> msg: arroyo_server_common: panicked at /home/zhuliquan/.cargo/registry/src/rsproxy.cn-8f6827c7555bfaf8/once_cell-1.19.0/src/lib.rs:1311:25:
Lazy instance has previously been poisoned panic.file="/home/zhuliquan/.cargo/registry/src/rsproxy.cn-8f6827c7555bfaf8/once_cell-1.19.0/src/lib.rs" panic.line=1311 panic.column=25
zhuliquan commented 4 weeks ago

Sorry, It's not bug. I found that your docker file missing feature profiling which can export heap pporf. Why not enable this feature?

https://github.com/ArroyoSystems/arroyo/blob/e5395fbe8288fc9cc119915bf58c88e7938faed6/crates/arroyo/Cargo.toml#L7

https://github.com/ArroyoSystems/arroyo/blob/e5395fbe8288fc9cc119915bf58c88e7938faed6/docker/Dockerfile#L49

mwylde commented 4 weeks ago

Heap profiling is disabled by default as it incurs some runtime overhead, and is mostly useful to developers working on arroyo and who can build their own binaries with it enabled. It makes sense to add an option to the dockerfile to allow it to be enabled though.