Linux Performance Analysis in 60,000 Milliseconds

[nikotung]

From here

Tools for performance analysis.

[nikotung]

uptime,from man

uptime gives a one line display of the following information. The current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes.

 uptime                                                                                                                                                                                                                                
 16:35:55 up 47 days,  3:13,  3 users,  load average: 0.13, 0.05, 0.01

1/5/15 分钟的CPU 平均负载

[nikotung]

dmesg | tail, detail from man

dmesg is used to examine or control the kernel ring buffer. The default action is to display all messages from the kernel ring buffer.

查看系统最新的十条日志消息。tail 默认是打印十条消息。主要是在日志中查看是否有错误或者告警类消息。

Print the last 10 lines of each FILE to standard output. With more than one FILE, precede each with a header giving the file name.

[nikotung]

vmstat, detail from man(8)

 vmstat reports information about processes, memory, paging, block IO, traps, disks and cpu activity.

  The first report produced gives averages since the last reboot.
  Additional reports give information on a sampling period of
  length delay.  The process and memory reports are instantaneous
  in either case.

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 6  0 5599076 4429268 891984 8557236    0    0    32  3625    1    1 17  7 76  0  0
 3  0 5599076 4408840 891984 8578024    0    0     0     0 4648 28036 23 12 65  0  0
 2  0 5599076 4418080 891984 8568928    0    0     0     0 4613 28030 20 11 69  0  0
 4  0 5599076 4417928 891984 8568460    0    0     0   160 4572 114071 17 10 73  0  0
 1  0 5599076 4593392 891984 8566412    0    0     0     0 4771 171922 14  7 80  0  0
 1  0 5599076 4616244 891984 8566348    0    0     0     0 4242 172925 11  6 82  0  0
 1  0 5599076 4576048 891984 8565836    0    0     0   348 4030 134351 12  5 83  0  0
 1  0 5599076 4528100 891984 8565624    0    0     0     0 4153 143829 12  5 83  0  0
 1  0 5599076 4627920 891984 8565776    0    0     0    68 4380 235239 12  7 81  0  0
 2  1 5599076 4598324 891984 8565776    0    0     0    32 4009 101302 11  3 85  0  0

进程部分的 r 和 b 数据

• r: The number of runnable processes (running or waiting for run time).
• b: The number of processes blocked waiting for I/O to complete.

swap 部分

• si, so: Swap­ins and swap­outs. If these are non­zero, you’re out of memory 非0 则代表没内存了。

CPU 部分

• us: Time spent running non-kernel code. (user time, including nice time)
• sy: Time spent running kernel code. (system time)
• id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
• wa: Time spent waiting for IO. Prior to Linux 2.5.41, included in idle.
• st: Time stolen from a virtual machine. Prior to Linux 2.6.11, unknown.

[nikotung]

pidstat detail from man

pidstat 1 10 # output 10 process stats with the interval of 1. 

The pidstat command is used for monitoring individual tasks currently being managed by the Linux kernel. It writes to standard output activities for every task selected with option -p or for every task managed by the Linux kernel if option -p ALL has been used. Not selecting any tasks is equivalent to specifying -p ALL but only active tasks (tasks with non-zero statistics values) will appear in the report.

Linux 3.10.0-957.5.1.el7.x86_64 (akuloan-platform-admin)        10/08/2021      _x86_64_        (4 CPU)

06:22:33 AM   UID       PID    %usr %system  %guest    %CPU   CPU  Command
06:22:34 AM  1024      4743    0.00    1.00    0.00    1.00     1  java
06:22:34 AM  1024      9885    0.00    1.00    0.00    1.00     0  java
06:22:34 AM  1028     23451    0.00    1.00    0.00    1.00     0  pidstat
06:22:34 AM     0     24808    1.00    0.00    0.00    1.00     3  python
06:22:34 AM     0     31886    1.00    1.00    0.00    2.00     2  AliYunDun

06:22:34 AM   UID       PID    %usr %system  %guest    %CPU   CPU  Command
06:22:35 AM  1024      7361    0.00    1.00    0.00    1.00     0  java
06:22:35 AM  1024      9885    1.00    0.00    0.00    1.00     0  java
06:22:35 AM     0     31886    1.00    1.00    0.00    2.00     2  AliYunDun

06:22:35 AM   UID       PID    %usr %system  %guest    %CPU   CPU  Command
06:22:36 AM  1024     17426    0.00    1.00    0.00    1.00     3  java
06:22:36 AM  1028     23451    0.00    1.00    0.00    1.00     0  pidstat
06:22:36 AM     0     31886    0.00    1.00    0.00    1.00     2  AliYunDun
06:22:36 AM     0     32739    1.00    0.00    0.00    1.00     0  aliyun-service

Average:      UID       PID    %usr %system  %guest    %CPU   CPU  Command
Average:     1024      4743    0.00    0.33    0.00    0.33     -  java
Average:     1024      7361    0.00    0.33    0.00    0.33     -  java
Average:     1024      9885    0.33    0.33    0.00    0.67     -  java
Average:     1024     17426    0.00    0.33    0.00    0.33     -  java
Average:     1028     23451    0.00    0.67    0.00    0.67     -  pidstat
Average:        0     24808    0.33    0.00    0.00    0.33     -  python
Average:        0     31886    0.67    1.00    0.00    1.67     -  AliYunDun
Average:        0     32739    0.33    0.00    0.00    0.33     -  aliyun-service

其中报告中的%CPU 字段含义如下

%CPU Total percentage of CPU time used by the task. In an SMP environment, the task's CPU usage will be divided by the total number of CPU's if option -I has been entered on the command line.