P403n1x87 / austin

Python frame stack sampler for CPython
https://pypi.org/project/austin-dist/
GNU General Public License v3.0
1.89k stars 54 forks source link

No samples collected #211

Closed zdyj3170101136 closed 8 months ago

zdyj3170101136 commented 9 months ago

Description

[root@plat-sh-data-testing-common-pic-infer-demo001 ~]# austin -p 30932 -mem
              _   _
 __ _ _  _ __| |_(_)_ _
/ _` | || (_-<  _| | ' \
\__,_|\_,_/__/\__|_|_||_| 3.6.0 [gcc 9.4.0]
# austin: 3.6.0
# interval: 100
# mode: memory
# memory: 3879940

🐍 Python version: 3.7.7

^C

duration: 3605829

😣 No samples collected.

version

cmd:

austin -p 30932 -mem
 austin version
              _   _
 __ _ _  _ __| |_(_)_ _
/ _` | || (_-<  _| | ' \
\__,_|\_,_/__/\__|_|_||_| 3.6.0 [gcc 9.4.0]

❌ Cannot launch the given command or it terminated too quickly.

python version:

[root@plat-sh-data-testing-common-pic-infer-demo001 ~]# /proc/30932/exe --version
Python 3.7.7
P403n1x87 commented 9 months ago

@zdyj3170101136 Austin 3.6 has dropped support for CPython < 3.8. Please use Austin 3.5 if you need to profile CPython 3.7: pip install austin-dist~=3.5.0

zdyj3170101136 commented 9 months ago

pip install austin-dist~=3.5.0

not work, too.

i am sure that process 30932 is a python process and costing cpu.

[root@plat-sh-data-testing-common-pic-infer-demo001 ~]# austin -mem  -p  30932
              _   _
 __ _ _  _ __| |_(_)_ _
/ _` | || (_-<  _| | ' \
\__,_|\_,_/__/\__|_|_||_| 3.5.0 [GCC 9.4.0]
# austin: 3.5.0
# interval: 100
# mode: memory
# memory: 3879940

🐍 Python version: 3.7.7

^C
# duration: 21342104

😣 No samples collected.
P403n1x87 commented 9 months ago

@zdyj3170101136 could you check that samples are collected in either wall time or CPU time mode, please? The commands would be

austin -p  30932

and

austin -sp  30932

respectively. Also note that the short option for memory is -m, not -mem.

Please note that, if the application is running but not allocating enough memory to cause changes in the resident memory levels, then Austin would not collect any samples. If you are interested in allocations rather than RSS, perhaps you could try the memory mode of Echion (note that this is still an experimental project). The output is the same as Austin in binary mode, i.e. MOJO. You could use the mojo2austin command from the austin-python package to convert to Austin non-binary format if needed.

zdyj3170101136 commented 9 months ago

@zdyj3170101136您能检查一下样本是在 Wall Time 模式还是 CPU Time 模式下收集的吗?命令是

austin -p  30932

austin -sp  30932

分别。另请注意,内存的简短选项是-m, not -mem

请注意,如果应用程序正在运行但没有分配足够的内存来导致驻留内存级别发生变化,则 Austin 将不会收集任何样本。如果您对分配而不是 RSS 感兴趣,也许您可​​以尝试Echion的内存模式(请注意,这仍然是一个实验项目)。输出与二进制模式下的Austin相同,即MOJO。如果需要,您可以使用austin-pythonmojo2austin包中的命令转换为 Austin 非二进制格式。

i think the austin output is not correct.

my benchmark.py read lots of file and call image.tobytes tran them to bytes.

austin shows that only 30% of memory account for func tobytes, which is 90% percent in actual.

austin -m pip  -o b python3  benchmakr.py

cat b | sed '/^#/d' | FlameGraph/flamegraph.pl --countname=μs > test.svg

截屏2024-01-22 下午12 47 57

zdyj3170101136 commented 9 months ago

@zdyj3170101136您可以检查一下样本是在 Wall Time 模式还是 CPU Time 模式下收集的吗?命令是

austin -p  30932

austin -sp  30932

分别。另外请注意,内存的简短选项是-m,而不是-mem。 请注意,如果应用程序正在运行但没有分配足够的内存来导致驻留内存级别发生变化,则 Austin 将不会收集任何样本。如果您对分配而不是 RSS 感兴趣,也许您可​​以尝试Echion的内存模式(请注意,这仍然是一个实验项目)。输出与Austin下的二进制模式相同,即MOJO。如果需要,您可以使用austin-pythonmojo2austin包中的命令转换为Austin非二进制格式。

我认为奥斯汀的输出不正确。

我的 benchmark.py 读取了大量文件并调用 image.tobytes 将它们转换为字节。

austin 显示只有 30% 的内存占 func tobytes,实际占 90%。

austin -m pip  -o b python3  benchmakr.py

cat b | sed '/^#/d' | FlameGraph/flamegraph.pl --countname=μs > test.svg

截屏2024-01-22 下午12 47 57

i use memray and scalene to test.

and the output is correct: image

P403n1x87 commented 9 months ago

@zdyj3170101136 please note that Austin and memray/scalene measure different things. Austin is tracking the resident memory and making a best guess as to which call stack was responsible for any variations (positive or negative). Memray/scalene trace memory allocations and attach them the actual stack that made those allocations. The "right" tool to use here depends on what one is trying to measure. Note that a memory allocation does not necessarily imply an increased usage in resident memory (e.g. because of COW optimisations etc...)