Releasing The Potential Of GNU Core Utilities, Part 3

There are numerous great uses for each of the Linux coreutils, and it’s difficult to choose which ones we will cover. Text utils are by far my favorite and they hold some of the most common practical uses. Let’s look at how practical they are and, with a bit of creativity, how powerful.

Text Utils

For the sake of brevity, let’s take just one example and use as many of the text utils as we can. We will use: cat, tac, sort, head, tail, cut, paste, and nl. It may seem like a lot, but let’s say you have a todo list in a csv format that looks like the following:

Month,Year,Who,Task
Feb,2011,Larry,Finalize project details
Jan,2011,Jim,Discuss project
May,2011,Bob,Fund raising event
Apr,2011,Tim,Plan fund raising event
Mar,2011,Everyone,Launch project

Now, let’s say you want to know how many items are on the todo list. We can print the list to standard out and use the word count command as follows:

$ cat todo.csv | tail -n +2 | wc -l
5

The `cat` command prints the file to standard out then uses the `|` “pipe” to send that as standard in to the next command. The next command is `tail` which with the `-n +2` flag it cuts out the first line, which is our field headings. Finally, we send that as standard in to our word count command, `wc`, that gives us the number of lines, 5, as requested by the `-l` flag.

Now, instead, let’s say you want to know just the month and the task, but display them in a numbered list:

$ cat todo.csv | tail -n +2 | cut -d "," -f 1,4 --output-delimiter=", " | sort -M | nl -w 2 -s ".) "
 1.) Jan, Discuss project
 2.) Feb, Finalize project details
 3.) Mar, Launch project
 4.) Apr, Plan fund raising event
 5.) May, Fund raising event

There’s a lot there, but only three commands are added. The `cut` command is super handy. If you have used PHP, you can think of it like the explode function, only you choose which fields you want. So, here we are choosing field 1 and 4 and then adding a modified output delimiter flag to make it look prettier (more spacing). Next, we sort the list by month using the `-M` flag. Finally, we add numbers to our list. The `-w 2` flag simply means how many characters wide the max number will be and the `-s “.) “` flag adds a suffix to the number.

Finally, let’s say you want a list of everyone who will be working with you on these tasks. Using the paste command with the previous modifiers we can do the following:

$ cat todo.csv | tail -n +2 | cut -d "," -f 3 | nl -w 2 -s ".) " | paste -d "," -s
 1.) Larry, 2.) Jim, 3.) Bob, 4.) Tim, 5.) Everyone

The `paste` command is just as handy as `cut`, especially if you ever need to create CSV files from individual files. Here, if we didn’t use paste we would get a vertical column of text. Instead, we want a list so that we can perhaps email it to a coworker. Thus, using the `-s` flag we get just what we are looking for.

Other Examples

Here are a couple other examples to try out. Test your knowledge and see if you can figure out what they are doing before you try them:

cat /etc/group | cut -d ":" -f 1 | sort -r | nl -w 3 -s ".) " | paste -d "," -s

cat /etc/passwd | cut -d ":" -f 1 | sort -r | nl -w 3 -s ".) " | paste -d "," -s

tac /etc/lsb-release | head -n 1 | cut -d "=" -f 2

One can use combinations of these commands to do anything from extract data from spreadsheets to inventory servers. Having these tools in your work bag can pay off in unexpected moments.