Text Processing — Searching and manipulating text
awk
Text Processing
Pattern scanning and processing language
Syntax
awk 'PROGRAM' [FILE...]
Examples
awk '{print $1}' file.txt
awk -F: '{print $1}' /etc/passwd
awk '{sum += $1} END {print sum}' nums.txt
Common Flags
-F set field separator
Notes
$1, $2... refer to fields. $0 is the whole line. NR is the line number.
cut
Text Processing
Remove sections from each line of input
Syntax
cut [OPTIONS] [FILE...]
Examples
cut -d: -f1 /etc/passwd cut -c1-10 file.txt
Common Flags
-d delimiter
-f field numbers
-c character positions
diff
Text Processing
Compare files line by line
Syntax
diff [OPTIONS] FILE1 FILE2
Examples
diff file1.txt file2.txt diff -u old.py new.py
Common Flags
-u unified format (easier to read)
-r compare directories recursively
-i ignore case
grep
Text Processing
Search text using patterns
Syntax
grep [OPTIONS] PATTERN [FILE...]
Examples
grep "error" /var/log/syslog grep -r "TODO" ./src grep -i "warning" app.log grep -n "def " app.py
Common Flags
-r recursive search
-i case-insensitive
-n show line numbers
-l only print filenames
-v invert match
-c count matches
-E extended regex
sed
Text Processing
Stream editor for filtering and transforming text
Syntax
sed [OPTIONS] SCRIPT [FILE...]
Examples
sed 's/foo/bar/g' file.txt sed -i 's/old/new/g' file.txt sed -n '5,10p' file.txt
Common Flags
-i edit file in-place
-n suppress default output
s/old/new/g substitute all occurrences
Notes
The -i flag modifies the file directly — always back up first or test without -i.
sort
Text Processing
Sort lines of text
Syntax
sort [OPTIONS] [FILE...]
Examples
sort file.txt sort -r file.txt sort -n numbers.txt sort -u file.txt
Common Flags
-r reverse order
-n numeric sort
-u remove duplicates
-k N sort by field N
-t set field delimiter
wc
Text Processing
Count lines, words, and characters
Syntax
wc [OPTIONS] [FILE...]
Examples
wc -l file.txt wc -w file.txt grep "error" log.txt | wc -l
Common Flags
-l count lines
-w count words
-c count bytes
-m count characters