Shell

[ahasusie]

shell recognizes three fundamental kinds of commands: built-in commands shell functions external commands: shell runs by creating a separate process

1. Create a new process. This process starts out as a copy of the shell.
2. In the new process, search the directories listed in the PATH variable for the given command. /bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin might be a typical value of PATH. (The path search is skipped when a command name contains a slash character, /.)
3. In the new process, execute the found program by replacing the running shell program with the new program. When the program finishes, the original shell continues by reading the next command from the terminal

sometimes the unix may contain multiple shells, it needs to tell which shell to run, so start with #! /bin/sh

[ahasusie]

/dev/null This is useful when you need a command's exit status but not its output

---

? need check ? The shell looks for commands in each directory in $PATH. It's common to have a personal bin directory in which to store your own private programs and scripts, and to list it in PATH by doing an assignment in your .profile file.

[ahasusie]

text processing https://www.gnu.org/software/sed/manual/sed.html grep, re, sed, awk, sort, uniq, wc, head, tail, tr, cut, join, cmp, diff, type, locate, find, df, du, ls, stat, touch, ps, top, kill, less

---

cut, join: working with fields

cut -d ":" -f 2 test.txt
cut -b 2-5 test.txt

sed: only for substitution differences between line and string

sed 's/stringA/stringB/X' testfile      
/2, replace the second occurrence of a pattern in a line
/g, replace all the occurrence of the pattern in a line, g means global(in a line)
/ng, replace from nth occurrence of the pattern in a line
/p, print 
/d, delete. sed 'nd  $d  3,6d  12,$d  pattern/d' testfile

sed '3 s/unix/linux/' geekfile.txt
# replace on specified line number

sed -n 's/unix/linux/p' testfile
# only print the replaced lines

# shell substitution
$ word='are'
$ sed -n "/$word/p" poem.txt
Roses are red,
Violets are blue,
And so are you.

$ echo 'current working dir is: ' | sed 's|$|'"$(pwd)"'|'
current working dir is: /home/learnbyexample/command_line_text_processing

$ echo 'today is date' | sed 's/date/'"$(date +%A)"'/'
today is Tuesday

awk: -F fs To specify a file separator. -f file To specify a file that contains awk script. -v var=value To declare a variable.

$ awk -F: '{print $1}' /etc/passwd      # -F sets the sperator, $n for the nth field
{print $1 " home at " $6}                   # testfile
$ awk -F: -f testfile /etc/passwd

export export command, which puts variables into the environment. The environment is simply a list of name-value pairs that is available to every running program. New processes inherit the environment from their parent, and are able to modify it before creating new child processes of their own. The export command adds new variables to the environment.

[ahasusie]

regular expression

match() if matched, return a matched object. search() if searched, return a matched object.

findall() Find all substrings where the RE matches, and returns them as a list. finditer() Find all substrings where the RE matches, and returns them as an iterator. split() sub()

dealing with the match objects group() Return the string matched by the RE start() Return the starting position of the match end() Return the ending position of the match span() Return a tuple containing the (start, end) positions of the match

# p = re.compile('[a-z]+')
# m = p.match(str)
# if m:
#     print m.group()
# else:
#     print "not match"

# >>> p = re.compile('[a-z]+')
# >>> m = p.search('::: message'); print(m)
# <_sre.SRE_Match object at 0x7ffe0d6f86b0>
# >>> m.group()
# 'message'
# >>> m.span()
# (4, 11)

# >>> p = re.compile(r'\d+')
# >>> p.findall('12 drummers drumming, 11 pipers piping, 10 lords a-leaping')
# ['12', '11', '10']

# >>> iterator = p.finditer('12 drummers drumming, 11 ... 10 ...')
# >>> iterator  
# <callable_iterator object at 0x...>
# >>> for match in iterator:
# ...     print(match.span())
# ...
# (0, 2)
# (22, 24)
# (29, 31)

txt = "monkey see monkey do, call me monkey in the zoo"
pattern = "monkey"
reg = re.compile(pattern)

x = reg.match(txt)
if x:
    print x.group()

x = reg.search(txt)
if x:
    print x.group()
    print x.span()

x = reg.findall(txt)
print x

x = re.split("\s", txt)
print x
x = re.sub("\s", "9", txt, 2)
print x

[ahasusie]