search

Tencent / Tendis

Tendis is a high-performance distributed storage system fully compatible with the Redis protocol.
http://tendis.cn
Other
2.92k stars 319 forks source link

【版本2.5.0】空闲情况下，CPU一直占用10%，是正常的吗？ #194

Closed lbp0200 closed 2 years ago

lbp0200 commented 2 years ago

试了2.4.3版本，空闲时CPU占用就很低

—————————————————————————————— 版本：tendisplus-2.5.0-rocksdb-v6.23.3

单机开发调试

配置文件

# tendisplus configuration for testing
bind 0.0.0.0
port 51002
daemon false
loglevel notice
logdir /var/local/tendis/log
dumpdir /var/local/tendis/dump
dir /var/local/tendis/db
pidfile /var/local/tendis/tendisplus.pid
slowlog /var/local/tendis/log/slowlog
databases 1

binlog-enabled 0
rocks.blockcachemb 128
rocks.compress_type lz4
raffertyyu commented 2 years ago

我用你的配置试了一下，我自己电脑上2.4.3、2.5.0都没有复现这个问题，启动后空闲状态CPU使用大概3-4%，用redis benchmark简单压了一下，cpu使用上升，然后结束测试负载2s之后cpu使用就恢复到3%了。

我本地机器配置是16线程32g，可以分享一下你的系统配置，或者可以附上火焰图嘛。也可以使用top -H -p 加上tendis的进程号来看具体是哪个线程的占用，如果可以的话，希望能回复一下，方便确认问题。

lbp0200 commented 2 years ago

电脑是Mac，用Parallels Desktop搭建2核512兆内存虚拟机，系统Ubuntu 20.04，只运行tendis 2.5.0版 top -H -p 593

top - 12:07:38 up 23:30,  2 users,  load average: 0.25, 0.24, 0.20
Threads:  55 total,   0 running,  55 sleeping,   0 stopped,   0 zombie
%Cpu(s):  3.1 us,  3.9 sy,  0.0 ni, 93.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :    466.3 total,     27.7 free,    166.8 used,    271.8 buff/cache
MiB Swap:   1939.0 total,   1919.7 free,     19.2 used.    280.2 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                          
   1001 lbp       20   0  550756  39536   6464 S   4.0   8.3  54:02.40 tx-io-rw-0                                                                       
   1002 lbp       20   0  550756  39536   6464 S   3.7   8.3  54:28.59 tx-io-rw-1                                                                       
   1030 lbp       20   0  550756  39536   6464 S   2.3   8.3  30:23.49 tx-repl-loop                                                                     
   1029 lbp       20   0  550756  39536   6464 S   0.3   8.3   0:26.16 tx-log-recyc_4                                                                   
    593 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:03.12 tendisplus                                                                       
    621 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:low0                                                                     
    622 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:low1                                                                     
    623 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:low2                                                                     
    624 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:low3                                                                     
    625 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:low4                                                                     
    626 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:low5                                                                     
    627 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:low6                                                                     
    628 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:low7                                                                     
    629 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:high0                                                                    
    630 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 rocksdb:high1                                                                    
    665 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:03.17 tendisplus                                                                       
    992 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:13.94 tx-worker-0_0                                                                    
    993 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:13.89 tx-worker-0_1                                                                    
    994 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:13.86 tx-worker-0_2                                                                    
    995 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:13.59 tx-worker-0_3                                                                    
    996 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:21.94 tx-worker-4_0                                                                    
    997 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:21.71 tx-worker-4_1                                                                    
    998 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:22.01 tx-worker-4_2                                                                    
    999 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:21.70 tx-worker-4_3                                                                    
   1000 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:09.04 tx-io-accept                                                                     
   1003 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 tx-repl-minc_0                                                                   
   1004 lbp       20   0  550756  39536   6464 S   0.0   8.3   0:00.00 tx-repl-minc_1
lbp0200 commented 2 years ago

这是在公司服务器上运行，也是虚拟机，具体配置未知，使用htop看CPU也是10%左右，一起运行的还有我搭建的3台2.4.3版本的集群，CPU占用不到2%

top - 20:12:36 up 282 days, 10:26,  2 users,  load average: 1.22, 0.40, 0.14
Threads: 101 total,   0 running, 101 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.3 us,  0.4 sy,  0.0 ni, 99.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  8118068 total,   122648 free,  3122492 used,  4872928 buff/cache
KiB Swap:  8257532 total,  8256496 free,     1036 used.  4252252 avail Mem 

   PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                                                                             
 85696 liubopi+  20   0  973892  42316  11336 S  1.3  0.5   0:00.86 tx-io-rw-0                                                                          
 85697 liubopi+  20   0  973892  42316  11336 S  1.3  0.5   0:00.87 tx-io-rw-1                                                                          
 85699 liubopi+  20   0  973892  42316  11336 S  1.3  0.5   0:00.89 tx-io-rw-3                                                                          
 85698 liubopi+  20   0  973892  42316  11336 S  1.0  0.5   0:01.09 tx-io-rw-2                                                                          
 85700 liubopi+  20   0  973892  42316  11336 S  1.0  0.5   0:00.90 tx-io-rw-4                                                                          
 85701 liubopi+  20   0  973892  42316  11336 S  1.0  0.5   0:00.84 tx-io-rw-5                                                                          
 85702 liubopi+  20   0  973892  42316  11336 S  1.0  0.5   0:00.82 tx-io-rw-6                                                                          
 85703 liubopi+  20   0  973892  42316  11336 S  0.7  0.5   0:00.80 tx-io-rw-7                                                                          
 85345 liubopi+  20   0  973892  42316  11336 S  0.3  0.5   0:00.03 tendisplus                                                                          
 85731 liubopi+  20   0  973892  42316  11336 S  0.3  0.5   0:00.66 tx-repl-loop                                                                        
 85304 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.14 tendisplus                                                                          
 85305 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:low0                                                                        
 85306 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:low1                                                                        
 85307 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:low2                                                                        
 85308 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:low3                                                                        
 85309 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:low4                                                                        
 85310 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:low5                                                                        
 85311 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:low6                                                                        
 85312 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:low7                                                                        
 85313 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:high0                                                                       
 85314 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 rocksdb:high1                                                                       
 85647 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 tx-worker-0_0                                                                       
 85648 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 tx-worker-0_1                                                                       
 85649 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 tx-worker-0_2                                                                       
 85650 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 tx-worker-0_3                                                                       
 85651 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 tx-worker-0_4                                                                       
 85652 liubopi+  20   0  973892  42316  11336 S  0.0  0.5   0:00.00 tx-worker-0_5
raffertyyu commented 2 years ago

下面那个服务器上的使用率看起来比较正常，每个io线程普遍0.7-1%左右，和我们的测试结果差不多。mac虚拟机上这个，应该就和机器有关了。每个io线程4%看着确实有点问题。我之后有空的时候开下虚拟机试试看。请问是m1还是intel的。

lbp0200 commented 2 years ago

2017年的Macbook，Intel I7 7820HQ

lbp0200 commented 2 years ago

如果第二个没有问题，那就可能是我虚拟机的问题，我没有安装增强工具。