5.22 sed用法收集tips-1

vieyahn2017 commented 5 years ago

sed用法收集

vieyahn2017 commented 5 years ago

shell在文本第一行和最后一行添加字符串

https://blog.csdn.net/u010801696/article/details/78819343

sed '1i 添加的内容' file  #这是在第一行前添加字符串
sed '$i 添加的内容' file  #这是在最后一行行前添加字符串
sed '$a添加的内容' file  #这是在最后一行行后添加字符串

测试证明，这种sed，被添加的内容如果有空格，或者别的特殊符号，处理起来很麻烦。

在最后一行行后添加，直接>>重定向追加，才是最省事的

vieyahn2017 commented 5 years ago

Linux下在文件内部指定行（首行、末尾行等）插入内容

1、在文件的首行插入指定内容：

:~$ sed -i "1i#! /bin/sh -" a

执行后，在a文件的第一行插入#! /bin/sh -

2、在文件的指定行（n）插入指定内容：

:~$ sed -i "niecho "haha"" a

执行后，在a文件的第n行插入echo "haha"

3、在文件的末尾行插入指定内容：

用第二种方法也可以。一般实现：

:~$ echo “haha” >> a

执行后，在a文件的末尾行插入haha

vieyahn2017 commented 5 years ago

使用sed命令对文件中匹配到的行进行删除

https://blog.csdn.net/qq_23953717/article/details/84343997 1、删除空行命令：sed "/^$/d"

如果需要对源文件进行替换，则加入-i参数即可

命令：sed -i "/^$/d" log.txt

2、删除匹配上模式的行命令：sed -i "/pattern/d" log.txt，比如：sed -i "/npm/d" log.txt，可以删除带有npm的行

vieyahn2017 commented 5 years ago

全文替换

sed -i "s#ES_HOST=.*#ES_HOST=$es_server_ip#g" config/hbase_es_conf.sh

# 被替换的值或者待替换的值 如果有引号之类的，要格外注意
sed -i "s#G_ADDITION_PARAMS=\".*\"#G_ADDITION_PARAMS=\"$HBASE_ADDITION_PARAMS\"#g" $1

vieyahn2017 commented 5 years ago

最近实现的NTP 功能代码

function log()
{
    local timestamp=`date -d today +'%y-%m-%d %H:%M:%S'`
    echo "[${timestamp}]: $1" >> $LOG_FILE
}

function check_NTP_installed()
{
    log "Checking NTP service installed. "

    exist_ntpd=$(rpm -qa | grep "ntp-" |wc -l)
    if [ "$exist_ntpd" -eq 0 ];  then
        log "Error: Network time protocol(NTP) not installed. Please install NTP first."
        return 1
    fi
}

function configure_NTP ()
{
    if [ ! -z $ntp_server_ip ]; then
        if [ `grep -c  "#server 0.centos.pool.ntp.org iburst" /etc/ntp.conf` -eq '0' ]; then
            sed -i s/"server 0.centos.pool.ntp.org iburst"/"#server 0.centos.pool.ntp.org iburst"/g  /etc/ntp.conf
            sed -i s/"server 1.centos.pool.ntp.org iburst"/"#server 1.centos.pool.ntp.org iburst"/g  /etc/ntp.conf
            sed -i s/"server 2.centos.pool.ntp.org iburst"/"#server 2.centos.pool.ntp.org iburst"/g  /etc/ntp.conf
            sed -i s/"server 3.centos.pool.ntp.org iburst"/"#server 3.centos.pool.ntp.org iburst"/g  /etc/ntp.conf
        fi
        service_ip_num_begin=`echo $ntp_server_ip | grep -o ','|wc -l`
        service_ip_num=$(($service_ip_num_begin+1))
        for(( i=0;i<$service_ip_num;i++))
            do
                count=$(($i+1))
                service_ip=`echo $ntp_server_ip | awk -F ',' '{print $'$count'}'`
                if [ `grep -c  "server $service_ip" /etc/ntp.conf` -eq '0' ]; then
                    sed -i "/Please consider joining the pool ./a\server $service_ip" /etc/ntp.conf
                    service ntpd start
                fi
            done
    fi

}

function check_NTP()
{
    check_NTP_installed
    if [ $? -ne 0 ]
    then
        return 1
    fi

    configure_NTP

    local ALARM_OFFSET=300000
    log "Checking NTP service status."

    exist_ntpd=$(ps -ef | grep -w "ntpd" | grep -vc grep)
    if [ "$exist_ntpd" -eq 0 ]
    then
        log "Error: Network time protocol(NTP) not running. Please start NTP first."
        return 1
    fi

    local bigdata_offset=$(/usr/sbin/ntpq -nc 'as' |awk '{print $2}' |grep '[0-9]\{1,\}' |xargs -i /usr/sbin/ntpq -nc 'rv {} offset' |grep offset= |cut -d= -f2 |cut -d. -f1 |head -n 1)
    if [ -z "$bigdata_offset" ]; then
        log "Error: Can't get time deviation, please check if ntp service is normal."
        return 1
    fi

    if [ ${bigdata_offset#-} -ge $ALARM_OFFSET ]; then
        log "Warn:Currently, the time deviation is more than 5 minutes. Synchronize the time between nodes."
    fi

    return 0
} 

check_NTP

vieyahn2017 commented 5 years ago

其中的

sed -i "/Please consider joining the pool ./a\server $service_ip" /etc/ntp.conf

会把 server $service_ip 内容加入到/etc/ntp.conf中 Please consider joining the pool . 这一行之下

实现把目标的server加入ntp服务器的功能

vieyahn2017 commented 5 years ago

最近实现的add_hosts功能

add_hosts.sh

#! /bin/bash

if [ ! -e $1 ]; then
    echo "hosts file is needed, add hosts failed."
    exit 1
fi

cat $1 | while read line
do
    if [ `grep -c "^${line}$" /etc/hosts` -eq 0 ]; then
        hostname=`echo $line | awk '{print $2}'`
        if [ `grep -c "^[0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}\s${hostname}$" /etc/hosts` -ne 0 ];then
            sed -i "/^[0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}\s${hostname}$/d" /etc/hosts
        fi
        echo $line >> /etc/hosts
    fi
done

hosts是最普通的hosts列表文件

10.15.19.3 host01
10.15.19.4 host02
10.15.19.5 host03

执行 sh add_hosts.sh hosts
会把hosts文件夹加入到当前机器，存在同名的会替换

vieyahn2017 commented 5 years ago

最近实现的 pre_install检查日志部分

调用部分

touch check_`date +%y%m%d%k%M%S`.log
check_log_file=`ls | grep check*log`
root_pwd=`cat /usr/bak_file/pwd.sh | grep root_pwd | cut -d "=" -f 2`
sed -i "s#root_pwd=.*#root_pwd=$root_pwd#g" check.sh
sh check.sh >> $check_log_file
mv /usr1/pre_install/Vid/$check_log_file /usr/bak_file/check_logs/

vieyahn2017 commented 5 years ago

install_sshpass.sh

#! /bin/bash
tar -zxvf  sshpass-1.06.tar.gz -C  prepare/ > /dev/null
cd prepare/sshpass-1.06
./configure > /dev/null
make > /dev/null
make install > /dev/null
cd -

vieyahn2017 / shellv