Open bingoohuang opened 5 years ago
➜ rig git:(master) ps aux | head -5
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
bingoobjca 1522 13.0 1.4 5568692 240992 ?? S 一09上午 22:33.13 /Applications/iTerm.app/Contents/MacOS/iTerm2
bingoobjca 10559 4.1 0.0 4371836 4960 s002 S 10:13上午 0:03.01 -zsh
bingoobjca 1361 3.8 1.3 6239448 218640 ?? S 一09上午 25:19.07 /Applications/QQ.app/Contents/MacOS/QQ
bingoobjca 78181 2.8 9.7 179434016 1629116 ?? S 四10上午 58:29.96 /Applications/GoLand.app/Contents/MacOS/goland
See the ps man page for more info.
VSZ - Virtual Set Size
The Virtual Set Size is a memory size assigned to a process ( program ) during the initial execution. The Virtual Set Size memory is simply a number of how much memory a process has available for its execution.
RSS - Resident Set Size
As oppose to VSZ ( Virtual Set Size ), RSS is a memory currently used by a process. This is a actual number in kilobytes of how much RAM the current process is using.
简单的说VSZ代表的是虚拟记忆体的使用量,RSS代表实体记忆体的使用量。但是这里RSS的值,是包括shared memory的。Linux会把一些share library(lib*.so)载入记忆体中,而不同的process会共享部份的share library,也就是说这些share library只被Linux在载入一次,而process可以从重复引用。
➜ rig git:(master) ps aux | awk '$2 == 1522 { print $3,$4,$6,$0 }'
7.9 1.4 240720 bingoobjca 1522 7.9 1.4 5567532 240720 ?? S 一09上午 22:51.44 /Applications/iTerm.app/Contents/MacOS/iTerm2
ps aux | sort -rn -k 6 |head -n 5
➜ rig git:(master) ps aux | sort -rn -k 6 |head -n 5
bingoobjca 78181 3.0 9.7 179445460 1631240 ?? S 四10上午 59:12.60 /Applications/GoLand.app/Contents/MacOS/goland
bingoobjca 510 0.0 2.7 6215516 458620 ?? S 一09上午 59:10.09 /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
bingoobjca 48414 1.1 2.1 4814908 359972 ?? S 三11上午 13:47.57 /Applications/VirtualBox.app/Contents/MacOS/VBoxHeadless --comment centos-kerb_default_1560232592051_15800 --startvm 82973c29-834c-497e-94f9-217e2eb7d49b --vrde config
bingoobjca 1522 5.4 1.7 5706412 285892 ?? S 一09上午 22:56.76 /Applications/iTerm.app/Contents/MacOS/iTerm2
bingoobjca 75281 0.0 1.7 5710188 285860 ?? S 四08上午 4:20.78 /Applications/WeChat.app/Contents/MacOS/WeChat
The TLDR pages are a community effort to simplify the beloved man pages with practical examples. tldr tar
# bingoo @ 192 in ~/github/dog on git:master o [11:57:37]
$ tldr tar
tar
Archiving utility.
Often combined with a compression method, such as gzip or bzip.
More information: <https://www.gnu.org/software/tar>.
- Create an archive from files:
tar -cf target.tar file1 file2 file3
- Create a gzipped archive:
tar -czf target.tar.gz file1 file2 file3
- Extract an archive in a target directory:
tar -xf source.tar -C directory
- Extract a gzipped archive in the current directory:
tar -xzf source.tar.gz
- Extract a bzipped archive in the current directory:
tar -xjf source.tar.bz2
- Create a compressed archive, using archive suffix to determine the compression program:
tar -caf target.tar.xz file1 file2 file3
- List the contents of a tar file:
tar -tvf source.tar
- Extract files matching a pattern:
tar -xf source.tar --wildcards "*.html"
list files order by size desc:
[betaoper@beta-hetong default]$ find . -type f -exec du -h {} + | sort -r -h |head -n 10
16M ./2019-07-09/1148515944367390722Convert.PDF
12M ./2019-06-28/1144528088150097921.PDF
7.9M ./2019-07-02/1146051722958139394Convert.PDF
7.2M ./2019-06-28/1144476741342109698.PDF
5.2M ./2019-07-01/1145605641686265857.PDF
4.6M ./2019-07-02/1146051549095849986.PDF
4.6M ./2019-07-02/1146050677066493953.PDF
4.1M ./2019-07-02/1146057879357550594.DOCX
4.1M ./2019-07-02/1146053558574313475.DOCX
3.5M ./2019-07-04/1146604946106630145.DOCX
指定用户名,进行svn操作: svn co URL --username xxx --password xxx
查看linux中某个端口(port)是否被占用
lsof lsof -i:3306
netstat -anp|grep 3306
查看局域网IP列表
Install nmap,
sudo apt-get install nmap
then
nmap -sP 192.168.1.*
or more commonly
nmap -sn 192.168.1.0/24
will scan the entire .1 to .254 range
This does a simple ping scan in the entire subnet to see which hosts are online.
$ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255
ether 52:54:00:ef:16:bd txqueuelen 1000 (Ethernet)
RX packets 8929283010 bytes 3287811980423 (2.9 TiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8071523788 bytes 3525672558442 (3.2 TiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1 (Local Loopback)
RX packets 2761872043 bytes 1133502059424 (1.0 TiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2761872043 bytes 1133502059424 (1.0 TiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
$ nmap -sP 192.168.1.*
Starting Nmap 6.40 ( http://nmap.org ) at 2019-11-26 11:01 CST
Nmap scan report for hadoop3 (192.168.1.3)
Host is up (0.00030s latency).
Nmap scan report for 192.168.1.5
Host is up (0.00080s latency).
Nmap scan report for hadoop1 (192.168.1.7)
Host is up (0.000088s latency).
Nmap scan report for 192.168.1.8
Host is up (0.00044s latency).
Nmap scan report for 192.168.1.9
Host is up (0.00042s latency).
Nmap scan report for hadoop2 (192.168.1.11)
Host is up (0.00060s latency).
Nmap scan report for hadoop0 (192.168.1.14)
Host is up (0.00031s latency).
Nmap scan report for 192.168.1.16
Host is up (0.00055s latency).
Nmap done: 256 IP addresses (8 hosts up) scanned in 2.41 seconds
Try following steps:
$ arp -a
? (169.254.0.4) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (169.254.0.74) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (169.254.1.12) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (172.18.0.6) at 02:42:ac:12:00:06 [ether] on br-8983f91a1c88
? (172.17.0.2) at 02:42:ac:11:00:02 [ether] on docker0
? (169.254.1.5) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (169.254.0.23) at fe:ee:ec:9b:c5:79 [ether] on eth0
hadoop0 (192.168.1.14) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (169.254.1.17) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (172.18.0.8) at 02:42:ac:12:00:08 [ether] on br-8983f91a1c88
? (172.18.0.3) at <incomplete> on br-8983f91a1c88
? (169.254.0.15) at fe:ee:ec:9b:c5:79 [ether] on eth0
gateway (192.168.1.1) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (172.18.0.10) at 02:42:ac:12:00:0a [ether] on br-8983f91a1c88
? (169.254.1.2) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (169.254.0.2) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (192.168.1.10) at fe:ee:ec:9b:c5:79 [ether] on eth0
hadoop3 (192.168.1.3) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (169.254.0.55) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (169.254.0.3) at fe:ee:ec:9b:c5:79 [ether] on eth0
hadoop2 (192.168.1.11) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (192.168.1.118) at fe:ee:ec:9b:c5:79 [ether] on eth0
? (172.18.0.7) at <incomplete> on br-8983f91a1c88
grep a file, but show several surrounding lines?
For BSD or GNU grep you can use -B num to set how many lines before the match and -A num for the number of lines after the match.
grep -B 3 -A 2 foo README.txt
If you want the same number of lines before and after you can use -C num.
grep -C 3 foo README.txt
This will show 3 lines before and 3 lines after.
-A and -B will work, as will -C n (for n lines of context), or just -n (for n lines of context... as long as n is 1 to 9).
To build on @Livven's answer, here are some useful combinations.
tee /path/to/file <<EOF
${variable}
EOF
tee /path/to/file <<'EOF'
${variable}
EOF
tee /path/to/file <<-EOF
${variable}
EOF
tee -a /path/to/file <<EOF
${variable}
EOF
tee /path/to/file <<EOF >/dev/null
${variable}
EOF
sudo -u USER tee /path/to/file <<EOF
${variable}
EOF
#!/bin/bash
# bccrack.sh
cd "/Applications/Beyond Compare.app/Contents/MacOS"
if [[ -f BCompare.real ]]; then
exit 0
fi
mv BCompare BCompare.real
cat > BCompare <<'EOF'
#!/bin/bash
rm "/Users/$(whoami)/Library/Application Support/Beyond Compare/registry.dat"
"`dirname "$0"`"/BCompare.real $@
EOF
chmod +x BCompare
journalctl -u service-name.service
-u flag is short for --unitjournalctl -u service-name.service -b
to see only log messages for the current boot, -b is short for --bootjournalctl -u service-name
For things named journalctl --since "2018-08-30 14:10:10" --until "2018-09-02 12:05:50"
journalctl --since "-1h15min"
journalctl --since "2015-01-10" --until "2015-01-11 03:00"
journalctl --since yesterday
journalctl -n
显示最近的10个条目journalctl -n 20
显示最近的20个条目journalctl _PID=8088 -o json
journalctl --since 09:00 --until "1 hour ago"
journalctl -f
view a live log of new messages as they are collectedThanks:
Manipulating network interfaces, firewalling, and forwarding from Go.
Manipulating network interfaces, firewalling, and forwarding from Go.
The last few years of development on Linux have been exciting on the network front:
Best of all, we can automate everything using Go.
A pre-requisite to any funky networking is bringing the interface up. This is trivial from Go, thanks to the rtnetlink API, codified by vishvananda's fabulous netlink package.
// Get a structure describing the network interface.
localInterface, err := netlink.LinkByName("eth0")
if err != nil {
// handle error
}
// Give the interface an address of 192.168.1.1, on a
// network with a 255.255.255.0 mask.
ipConfig := &netlink.Addr{IPNet: &net.IPNet{
IP: net.ParseIP("192.168.1.1"),
Mask: net.CIDRMask(24, 32)
}}
if err = netlink.AddrAdd(localInterface, ipConfig); err != nil {
// handle error
}
// Setup the default route, so traffic that doesn't hit
// 192.168.1.(1-255) can be routed.
if err = netlink.RouteAdd(&netlink.Route{
Scope: netlink.SCOPE_UNIVERSE,
LinkIndex: localInterface.Attrs().Index,
Dst: &net.IPNet{IP: gatewayIP, Mask: net.CIDRMask(32, 32)},
}); err != nil {
// handle error
}
// Lastly, bring up the interface.
if err = netlink.LinkSetUp(localInterface); err != nil {
// handle error
}
Under the hood, the following is occurring:
If you want to read about the wire encoding for this & other netlink interfaces, have a look at Matt Layher's posts on the subject.
NFTables, the successor to IPTables, is a highly-configurable rules engine for processing packets. It is configured via a netlink interface, much like the example above.
nftables是新式的数据包过滤框架,从Linux内核3.13版本开始可用,旨在替代现用的iptables框架。Nftables HOWTO 中文翻译,更多nftables相比iptables到底改变了什么
First, some terminology:
The triangles represent chains. Conceptually, it makes sense to think of them as hooks.
So, if you want to filter out certain IPv4 traffic from the internet from everything, you would put your rules in a prerouting chain with hook type filter, in a table with a name of your choosing. The table would have an address family of IPv4.
Hopefully this section gives you enough information to fill in the opaque-sounding fields. The rest is pretty straightforward.
I've talked for long enough, lets get into some code shall we? We use the nftables package by stapelberg, which is shaping up to be the canonical Go package for nftables (I'm biased thou, I'm sending PRs there too).
c := &nftables.Conn{}
// Lets create our table first.
myTable := c.AddTable(&nftables.Table{
Family: nftables.TableFamilyIPv4,
Name: "myFilter",
})
// Now, we create a chain which we will add our filter
// rules to.
myChain := c.AddChain(&nftables.Chain{
Name: "myChain",
Table: myTable,
Type: nftables.ChainTypeFilter,
Hooknum: nftables.ChainHookInput,
Priority: nftables.ChainPriorityFilter,
})
// Lets add a rule to the chain that loads the source
// address, and compares it to a hardcoded IP.
c.AddRule(&nftables.Rule{
Table: myTable,
Chain: myChain,
Exprs: []expr.Any{
// payload load 4b @ network header + 12 => reg 1
&expr.Payload{
DestRegister: 1,
Base: expr.PayloadBaseNetworkHeader,
Offset: 12,
Len: 4,
},
// cmp eq reg 1 0x0245a8c0
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: net.ParseIP("192.168.1.53").To4(),
},
// [ immediate reg 0 drop ]
&expr.Verdict{
Kind: expr.VerdictDrop,
},
},
})
// Apply the above (commands are queued till a call to Flush())
if err := c.Flush(); err != nil {
// handle error
}
Lets break this down.
Creating the chain is where the fields get confusing. We create a chain in the table we previously created, with the following fields:
Filter
type - Because we want to filter packetsInput
chain (hook) - Because we want to drop packets before they enter the system (but not interfere with forwarded traffic)Filter
priority - I havent talked about priority yet, because you should seldom need to change it. Priorities other than Filter allows you to make your rules run before/after internal nftables operations. See the nftables wiki for more details.I've avoided writing about these till now as they are a little confusing at first. That said, the rule engine is one of the most powerful features NFTables has on offer.
An NFTables rule is composed of a series of expressions, which are evaluated from beginning to end. If the expression checks some condition, and the condition evaluates to false, the remainder of the expressions are not evaluated (and hence any actions in later expressions are not applied).
Other NFTables rules perform some action. For instance expr.Drop
instructs NFTables to drop the packet if it is evaluated (conversely, expr.Accept
tells NFTables to accept the packet). There are also actions that increment counters, actions that mangle packets, actions that mark the connection or NAT it, and many more. Lets break down the example we saw before:
// payload load 4b @ network header + 12 => reg 1
&expr.Payload{
DestRegister: 1,
Base: expr.PayloadBaseNetworkHeader,
Offset: 12,
Len: 4,
},
This expression loads 4 bytes (the size of an IPv4 address) from offset 12 of the packet's network header. If we look at the wikipedia page for an IPv4 packet, we can see that offset 12 is the source IP address. Neat!
There are 20, 32bit registers you can use to store state as your rule is evaluated. Register 0
specifies the verdict (ie: accept/drop packet) code, so don't use it unless you know what you are doing.
// cmp eq reg 1 0x0245a8c0
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: net.ParseIP("192.168.1.53").To4(),
},
The next blob of code compares the contents of register 1
, permitting the rule to continue evaluation if the register contents match (CmpOpEq
) the data (192.168.1.53
). This has the effect of aborting rule evaluation if the source IP of the packet is not 192.168.1.53.
The last expression expr.Verdict
simply drops the packet if the rule gets that far in evaluation (internally, this sets register 0
). Since only packets with a source IP of 192.168.1.53
get this far, the expressions in this rule collectively add up to drop packets from 192.168.1.53
.
There's heaps more to cover but this post is getting long.
Building advanced NFTables rules
- I'll explain how to come up with more complex rules, and provide a load of copypasta for the common use cases.NFTables counters & sets
- NFTables introduced the ability to use in-kernel lookup datastructures called sets, which makes most rules drastically simpler, & better support the common use-cases (blocking/allowing/forwarding sets of IPs/ports). Counters also let you track the throughput of your rules, both in terms of the number of packets and the number of bytes.Filtering process traffic with net namespaces
- In this post I'll go over network namespaces, how to use them, and walk through a network filter for select processes.Version | Maintainer | Released | Projected EOL |
---|---|---|---|
5.4 | Greg Kroah-Hartman & Sasha Levin | 2019-11-24 | Dec, 2021 |
4.19 | Greg Kroah-Hartman & Sasha Levin | 2018-10-22 | Dec, 2020 |
4.14 | Greg Kroah-Hartman & Sasha Levin | 2017-11-12 | Jan, 2024 |
4.9 | Greg Kroah-Hartman & Sasha Levin | 2016-12-11 | Jan, 2023 |
4.4 | Greg Kroah-Hartman & Sasha Levin | 2016-01-10 | Feb, 2022 |
3.16 | Ben Hutchings | 2014-08-03 | Jun, 2020 |
CentOS version | Architectures | RHEL base | Kernel | CentOS release date | RHEL release date | Delay (days) |
---|---|---|---|---|---|---|
7.5-1804 | x86-64 | 7.5 | 3.10.0-862 | 10 May 2018 | 10 April 2018 | 31 |
7.6-1810 | x86-64 | 7.6 | 3.10.0-957 | 3 December 2018 | 30 October 2018 | 34 |
7.7-1908 | x86-64 | 7.7 | 3.10.0-1062 | 17 September 2019 | 6 August 2019 | 42 |
8.0-1905 | x86-64, POWER8 (le), AArch64 | 8.0 | 4.18.0-80 | 24 September 2019 | 7 May 2019 | 140 |
sed -n '/matched/,$p' file
awk '/matched/,0' file
In Awk there is , -- the "between" operator(range patterns). Following some examples (default action = print)
awk '/start/,/stop/'
awk 'NR==3, NR==10'
awk '/<ACROSS>/, 0'
(0 is false, so it never ends)you can freely mix regexps and line number predicates (or any expression) in them. For example:
NR==1,/rex/
# all lines from the 1rst up to the one matching /rex//rex/,0
# from the line matching /rex/ up to the end-of-file.