heidsoft / cloud-bigdata-book

write book
56 stars 33 forks source link

shell使用 #23

Open heidsoft opened 7 years ago

heidsoft commented 7 years ago

http://yysfire.github.io/linux/sed-usage-summary.html http://zhengheng.me/2015/11/12/sed-course/ http://coolshell.cn/articles/9104.html http://sed.sourceforge.net/sed1line_zh-CN.html http://man.linuxde.net/sed http://wiki.jikexueyuan.com/project/shell-learning/sed-search-and-replace.html http://coolshell.cn/articles/9104.html

heidsoft commented 7 years ago

http://www.cnblogs.com/chengmo/archive/2010/10/02/1841355.html http://www.cnblogs.com/emanlee/archive/2013/09/07/3307642.html

heidsoft commented 7 years ago

http://blog.chinaunix.net/uid-8656705-id-2017937.html

heidsoft commented 5 years ago

目录判断

if [ -d "$WORKING_DIR" ]; then rm -Rf $WORKING_DIR; fi
if [ $( ls <file> ) ]; then rm <file>; fi
#!/bin/sh

if [ -fe FILE ]
then 
    rm FILE
fi 

https://stackoverflow.com/questions/4846007/check-if-directory-exists-and-delete-in-one-command-unix

判断目录是否存在

General syntax to see if a directory exists or not
[ -d directory ]
OR
test directory
See if a directory exists or not with NOT operator:
[ ! -d directory ]
OR
! test directory

Find out if /tmp directory exists or not
Type the following command:
$ [ ! -d /tmp ] && echo 'Directory /tmp not found'

OR
$ [ -d /tmp ] && echo 'Directory found' || echo 'Directory /tmp not found'

#!/bin/bash
DIR="$1"

if [ $# -ne 1 ]
then
    echo "Usage: $0 {dir-name}"
    exit 1
fi

if [ -d "$DIR" ]
then
    echo "$DIR directory  exists!"
else
    echo "$DIR directory not found!"
fi

https://www.cyberciti.biz/tips/find-out-if-directory-exists.html

heidsoft commented 5 years ago

ps 显示内存排序

ps aux --sort -rss
ps aux --sort rss

ps 根据pid 排序

ps aux --sort pid

ps 显示信息案例

EXAMPLES
To see every process on the system using standard syntax:
   ps -e
   ps -ef
   ps -eF
   ps -ely

To see every process on the system using BSD syntax:
   ps ax
   ps axu

To print a process tree:
   ps -ejH
   ps axjf

To get info about threads:
   ps -eLf
   ps axms

To get security info:
   ps -eo euser,ruser,suser,fuser,f,comm,label
   ps axZ
   ps -eM

To see every process running as root (real & effective ID) in user format:
   ps -U root -u root u

To see every process with a user-defined format:
   ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
   ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
   ps -eopid,tt,user,fname,tmout,f,wchan

Print only the process IDs of syslogd:
   ps -C syslogd -o pid=

Print only the name of PID 42:
   ps -p 42 -o comm=

https://alvinalexander.com/linux/unix-linux-process-memory-sort-ps-command-cpu https://alvinalexander.com/unix/man/linux-ps-man-page-ps-help

ps 实时查看

watch -n 1 'ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head'

ps-command-examples-for-linux-process-monitoring

heidsoft commented 5 years ago

grep 高级技艺

grep 同时满足多个关键字
① grep -E "word1|word2|word3" file.txt 
满足任意条件(word1、word2和word3之一)将匹配。 
② grep word1 file.txt | grep word2 |grep word3 
必须同时满足三个条件(word1、word2和word3)才匹配。

grep 同时排除多个关键字
不说废话, 例如需要排除 abc.txt 中的 mmm nnn
grep -v 'mmm\|nnn' abc.txt 

grep '[0-9]:[0-9][0-9]:[0-9][0-9]'
If you need get timestamp only, and your grep is gnu grep.

grep -o '[0-9]:[0-9][0-9]:[0-9][0-9]'
and if you work more harder, limit on time format only:

grep '[0-2][0-9]:[0-5][0-9]:[0-5][0-9]'
heidsoft commented 5 years ago

数组使用

https://www.cyberciti.biz/faq/finding-bash-shell-array-length-elements/ https://stackoverflow.com/questions/10586153/split-string-into-an-array-in-bash https://stackoverflow.com/questions/15105135/bash-capturing-output-of-awk-into-array/15105237 https://www.cyberciti.biz/faq/finding-bash-shell-array-length-elements/

heidsoft commented 5 years ago

杀死进程

https://www.howtoing.com/how-to-kill-a-process-in-linux

heidsoft commented 3 years ago

Best Practices for Writing Bash Scripts

https://kvz.io/bash-best-practices.html https://yoone.eu/article/good-practices-for-writing-shell-scripts/

Use long options (logger --priority vs logger -p). If you're on cli, abbreviations make sense for efficiency. but when you're writing reusable scripts a few extra keystrokes will pay off in readability and avoid ventures into man pages in the future by you or your collaborators.

Use set -o errexit (a.k.a. set -e) to make your script exit when a command fails.

Then add || true to commands that you allow to fail.

Use set -o nounset (a.k.a. set -u) to exit when your script tries to use undeclared variables.

Use set -o xtrace (a.k.a set -x) to trace what gets executed. Useful for debugging.

Use set -o pipefail in scripts to catch mysqldump fails in e.g. mysqldump |gzip. The exit status of the last command that threw a non-zero exit code is returned.

!/usr/bin/env bash is more portable than #!/bin/bash.

Avoid using #!/usr/bin/env bash -e (vs set -e), because when someone runs your script as bash ./script.sh, the exit on error will be ignored.

Surround your variables with {}. Otherwise bash will try to access the $ENVIRONMENT_app variable in /srv/$ENVIRONMENT_app, whereas you probably intended /srv/${ENVIRONMENT}_app.

You don't need two equal signs when checking if [ "${NAME}" = "Kevin" ].

Surround your variable with " in if [ "${NAME}" = "Kevin" ], because if $NAME isn't declared, bash will throw a syntax error (also see nounset).

Use :- if you want to test variables that could be undeclared. For instance: if [ "${NAME:-}" = "Kevin" ] will set $NAME to be empty if it's not declared. You can also set it to noname like so if [ "${NAME:-noname}" = "Kevin" ]

Set magic variables for current file, basename, and directory at the top of your script for convenience.

Summarizing, why not start your next bash script like this:

!/usr/bin/env bash

Bash3 Boilerplate. Copyright (c) 2014, kvz.io

set -o errexit set -o pipefail set -o nounset

set -o xtrace

Set magic variables for current file & dir

dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" file="${dir}/$(basename "${BASH_SOURCE[0]}")" base="$(basename ${file} .sh)" root="$(cd "$(dirname "${__dir}")" && pwd)" # <-- change this as it depends on your app

arg1="${1:-}" If you have additional tips, please share and I'll update this post.

heidsoft commented 3 years ago

cat pods1.list.log |tr -s '[:blank:]' ',' > pods2.list.cvs