The sed utility can be used to print the contents of a
file, substitute a line (or multiple lines), and then save the file. In
contrast to grep, sed can substitute a line or multiple lines in a file and perform an in-place update of that file.
The simplest sed invocation when substituting foo for bar is:
$ sed 's/foo/bar/' inputfileExample: Remove comments
While grep can format output on screen, it is unable to modify a file in place. To do this, you’d need a file editor like ed . Since ed is not part of this article, use sed to achieve the same thing you did with grep in the previous article’s first example. This time modify the /etc/fstab file in-place passing the -i flag to sed . Without the -i flag , you’d only see what would have been modified.
You are encouraged to always run sed without the -i flag, just to make sure that the result it produces is expected. The sed utility also offers the -i.bak flag, which creates a backup file before editing.
The final grep command for this example was:
$ grep -v '^#' /etc/fstab > ~/fstab_without_commentWith sed, you have:
# sed -i '/^#/d' /etc/fstab /dev/mapper/VGCRYPTO-ROOT / ext4 defaults 1 1 UUID=e9de0f73-xxxx-xxx /boot ext4 defaults 1 2 LABEL=SSD_SWAP swap swap defaults 0 0
Example: Print only /etc/passwd users
In the grep example, you printed only usernames from the /etc/passwd file with the following:
$ grep -Eo '^[a-zA-Z_-]+' /etc/passwdYou can do the same using sed as follows:
$ sed 's/^\([a-zA-Z_-]\+\).*/\1/' /etc/passwdIn the above example, you group a match by parentheses (), and then print the matched group with \1 (back-reference), which dictates the first group. For a second group, you’d use \2, and so on.
Example: Replace all foo with bar
In sed, you can search for a pattern and then replace
only the occurrence matching the pattern. To replace all occurrences in
the file inputfile1 from foo to bar globally, run:
$ sed -i '/foo/bar/g' inputfile1Example: Replace a single instance
Take the file inputfile2, which has the following contents:
hello world second line should be replaced this line should be replaced later
Say that you want to replace should with will, but only for the second line. This command breaks down as follows:
$ sed '/second/s/should/will/' inputfile2
| | | |
| | | with this pattern
| | this pattern
| substitute
Search for the pattern "second"
This output is sent to standard output rather than replacing the file’s contents. The result looks like this:
$ sed '/second/s/should/will/' inputfile2 hello world second line will be replaced this line should be replaced later
The sed command is case-sensitive. The following won’t work when you try to replace World with there:
$ echo "Hello World" | sed 's/world/there/' Hello World
GNU sed introduced a new flag, /I, which ignores the case and will perform the replacement with the same command:
$ echo "Hello World" | sed 's/world/there/I' Hello there
Example: Print a range of lines and quit
With sed, you can also print lines and quit after your
criteria are met. The following commands will print three lines and
quit. This command:
$ sed -n '1,3p' /etc/passwdis equivalent to:
$ sed '3q' /etc/passwdThe following would be wrong:
$ sed '1,3q' /etc/passwd # Wrong. You cannot quit three times
Example: Comment out uncommented lines
Regular expressions can also be used with sed, as demonstrated earlier. For example, you have the following small script:
$ cat test_script #/usr/bin/env bash this is the first comment This is another comment # this is a comment too echo "This is not a comment and should be echoed"
You now have to skip the first line, starting with #!/bin/bash, and comment out the third and fourth lines, but not the fifth because that line is already commented.
In sed, you can use something like the following:
$ sed '3,6s/^[^#]/# &/g' test_script #/usr/bin/env bash # this is the first comment # This is another comment # this is a comment too echo "This is not a comment and should be echoed"
In the above command, the following is performed:
3,6sdefines a range, from line three down to line six./^[^#]/matches everything that is a character and does not start with a hash (#)./# &/greplaces a part, in this case it puts a#in front of the line dictated by the&sign.
Example: Remove all digits
Different applications generate data in different formats. With sed, you can keep only the data you can use. For example, you have the following file (inputfile3) in this format:
foo1234 bar99128 baz2842 qux12953 discard39120
Maybe a program generated the wrong format, or it concatenated the
fields to one. What if you were only interested in keeping the alpha
characters and wanted to discard the digits? How would you achieve this
goal with sed?
The answer is probably easier than you think:
$ sed 's/\([a-z]*\).*/\1/' inputfile3 foo bar baz qux discard
Example: Change specific lines
Furthermore, sed can also handle ranges by pattern, which means you can specify a start and an end string and manipulate the range. For example:
$ cat inputfile4 hello world start of the comment another comment end of a comment dont comment this line nor this line
The following sed command will comment lines starting with start and ending with end:
$ sed '/start/,/end/ s/^/# /' inputfile4 hello world # start of the comment # another comment # end of a comment dont comment this line nor this line
Get rid of the empty lines as well.
$ sed '/start/,/end/ s/^/# /;/^$/d' inputfile4 hello world # start of the comment # another comment # end of a comment dont comment this line nor this line
There is much more to sed and its rich features. To be able to fully utilize sed’s abilities, please see its documentation page, which you can find here.
Use sed if you are receiving data from a pipeline, or want to manipulate data on the fly.
The sed command is scripted and it’s easy to learn to
perform basic operations. All you need is practice, especially with
regular expressions.