The Command Line III: Essential tips for living on the command line

Share

This section is a bit of an interlude in our unix journey. Instead of addressing concepts such as file systems and the like, in this section we are going to look at some must know tips to make your life easier on the command line. You should have noticed by now that there is a lot of typing involved in the command line and, as any good programmer would rather write a tool once than continually repeat a common chore, there are mechanisms available to us on the command line to reduce the typing load. The following tips are essential to efficiency on the command line and will make your life easier.

I get around – Keyboard shortcuts

Just like the graphical interface there are a number of keyboard shortcuts available on the command line. The ones we are going to look at here are mostly related to text editing but there are others that we will run into when looking at job control. The shortcuts listed below are specific to the Bash shell and are derived from the shortcuts used by a text editor known as Emacs. Text editors are a topic for another post but by way of introduction there are historically two text editors that were widely used in Unix — Emacs and vi (pronounced “vee eye”, not “six”, not “vye”). These two editors inspired what is known as the Editor Wars, a long running conflict between aficionados, or some would say geeks without much else to do, of one text editor over another. We are not going to get into this right now, except to say that the Emacs bindings are the default in Bash but, maybe to appease vi users, it can be configured to use the vi bindings. So if you have already developed a preference for one editor over another rest assured that Bash can accommodate you. Anyway, here are the default shortcuts:

 
Keystroke Action
Ctrl + a Go to the beginning of the line you are currently typing on.
Ctrl + e Go to the end of the line you are currently typing on.
Ctrl + l Clears the Screen, similar to the ‘clear’ command (use ‘man clear’ for more)
Ctrl + u Deletes everything up to the cursor. If you are at the end of the line, deletes the entire line.
Ctrl + k Deletes everything in front of the cursor. If you are at the start of the line will delete the entire line.
Ctrl + h Same as backspace
Ctrl + w Deletes the word before the cursor.
Ctrl + t Swaps the last two characters before the cursor.
Esc + t Swaps the last two words before the cursor.
Alt + f Move cursor forward one word on the current line.
Alt + B Move cursor backward one word on the current line.

Once you have mastered these shortcuts and developed the necessary muscle memory you will find these shortcuts very useful. For example, I am writing this post in emacs and so the shortcuts are the same as on the command line (as an aside, you spend a lot of time in a text editor once you start programming and having the command line shortcuts and your editor shortcuts the same is a big advantage as you don’t need to think when moving between them). When typing I am always hitting the first letter of the next word before I get to the space bar. So, for example, if I was trying to type `on the’ I will often type `ont ‘by mistake (something I did writing this very paragraph). This simple error can be corrected immediately by hitting `Ctrl + t’ as soon as it is made. The alternative is to use two backspaces, a space and the ‘t’ to get back to what I wanted to type. Likewise, my typing speed is sufficient that it is quicker for me to delete backwards a few words (`Ctrl + w’) when I have made a typo and then re-type the words. All this may seem a little anal, and you do have to train yourself to use these keys rather than automatically heading for the backspace key (or worse the mouse), but you will be genuinely surprised at how much more efficient you become using these shortcuts. One last thing, the position of the ‘control’ key on almost keyboards is in an awkward position for regular use. One trick to alleviate this problem is to `remap’ the caps lock key to become the ‘control’ key. If you do this then to hit the ‘control’ key a simple reach with your pinky is sufficient, rather than awkwardly reaching down and across a bit. There are plenty of guides on the web on how to do this in your various OSs, including OSX, Windows 7 and Linux.

Just what I was thinking – Command completion

One of the most important time savers on the command line is command completion. At the command line you can use the tab key to complete a word or path that you are typing, if there is an unambiguous completion then the shell will finish it for you. If there is ambiguity then the shell will not complete but if you push the ‘tab’ key again it will list all the possible completions for your perusal. Lets try it, enter the following at your command line and press tab once:

$ unam

Sweet, a saving of one ‘e’. Not that impressive but, and trust me in on this, once you start using the command line reaching for the tab key will become as unconscious a motion as breathing. It is particularly useful when trying to remember commands (that, you know, begins with a `d’, or something). It is also very useful when typing a long path name. Can’t remember quite what the directory your looking for is called? Hit the tab key twice and the directories in the last directory typed will be listed for you. It is pretty hard to visualise this so lets walk through a few examples. In the example above command completion finished `unam’ to the command `uname’. If you press return and did the exercises in the last section you will not be surprised to see the name of your operating system displayed by the shell. Now, enter the following at the command line and press the ‘tab’ key once.

$ un

This time nothing happened, except maybe your computer beeped at you. Now push the ‘tab’ key again and you should see something similar to the following:

$ un
unalias      uncompress   unflatten    unifdefall   
uname        unexpand     unifdef      uninstall    
unlink       unset        unvis        unzip  
$ un

What the shell has done is show you all the possible completions of a command that begins with the letters ‘un’ (I’ve truncated the display above to fit in the page). Now type and `a’ and hit the tab key twice. The list is reduced to those commands beginning with `una’ (only two commands on my system but this will vary depending on what programs you have installed). You can keep adding letters until you get an unambiguous completion, for `uname’ this will happen when we get to the ‘m’. As you use the command line more often you will get to the point where you know how much of a certain command you need to type before hitting the ‘tab’ key to get the rest inserted for you. This saves you a lot of typing. Command completion is also useful for when you are not sure exactly how to spell a command. Just enter the first few letters that you are sure of, hit ‘tab’ twice and look for the one you want. If you want to see how many commands are available try hitting the tab key twice at the blank prompt (use ‘q’ to get out of listing you may end up in).

The ‘tab’ key also works for paths. Go to your home directory (‘cd’) and if you haven’t deleted the ‘programming-4-biologists’ directory that we created in the last lesson (if you’ve deleted it make another one — ‘mkdir’) try moving into it using command completion. Type the following and push the ‘tab’ key:

$ cd prog

Either ‘programming-4-biologists’ will appear or you will get beeped at. If beeped at press the ‘tab’ key again and see what other file or directory is similar to ‘programming-4-biologists’ and add letters and press the ‘tab’ key until ‘programming-4-biologists’ does appear. Then press enter and check where you are with the ‘pwd’ command. One more time — try entering the following and then pressing the ‘tab’ key, what happens? Now push it again twice.

$ ls /var/lo

On the command line the tab key is one of those people who always finishes your sentences for you, the tab key, however, usually gets it right.

You can’t escape your past – Bash command line history

The shell keeps a log of all the commands that you have entered. This list even persists over multiple terminal session, ie. if you have closed your terminal and opened another one later the command history will still be there from your last session. This is super useful when entering long, frequently used commands that you just haven’t had time to put into a script or write an alias for (more on these later). Bash, by default, will retain the last 500 commands that you have typed and they are, again by default, stored in a hidden file (files beginning with a `.’ – covered in post 2) in your home directory called ‘.bash_history’. You can change these parameters but I will leave that as an exercise for those interested (this is a very good in-depth tutorial on the Bash history functions).

You can have a look at your command history by having a look at the history file by using the `less’ command. Text files are very common on a Unix system, they are used for configuration files, source code and all manner of other purposes. Because of this there are a whole suite of unix commands that lets one view the contents of a text file. The `less’ program is one of these programs. It is known as a ‘pager’ and it will present a text file on the terminal one page at a time. Whenever you have looked at a man page you have used something similar and pretty much the same navigation and exiting commands we used in navigating man pages can also be used in ‘less’ (see `man is your friend’ in the first post). So to have a look at your history file type the following (using command completion if you like) and press return:

$ less ~/.history_bash

You should see a list of the last 500 commands you have entered. You are being watched it appears. If paranoid you can delete this file every time your finished, or disable history logging, but most of us, while not up to nefarious deeds, find history logging quite useful (it is also likely on a multi-user system, your being logged by a higher power as well). Now it is nice having a list of commands but there are more useful ways to access this information. At the command line the ‘history’ command will return a the list of commands from your history file. You can also restrict the list to the last N (where N is a number) by using ‘history N’. You can also do other things with ‘history’, including clearing or deleting history items, and these are all listed in the `history’ man page (‘man history’ remember).

What if you just entered a long command but made a small error, do you need to type the whole command in again or go to the history file and copy it? No, you can scroll backward through your history using ‘Ctrl + p’ and forward using ‘Ctrl + n’. Try it, in your home directory enter the following and press enter:

$ ls

You should get a list of your home directory, or whichever directory you are in. Now press ‘Ctrl + p’, the last command should now be at your prompt waiting for editing or re-execution by pushing enter. Now push ‘Ctrl +p’ a number of times and see how your command history scrolls through your command line, use ‘Ctrl + n’ to move forwards once you have gone backwards a few times. All your commands are at you disposable and you can move backwards and forwards using ‘Ctrl + p’ and ‘Ctrl + n’ (almost always these commands are also bound to the arrow keys on your keyboard so you can do the same using the up and down arrows which is what most people normally use).

What you if you entered a command a long time ago? You could push ‘Ctrl + p’ three or four hundred times or you could use the search function. To try this out enter the following commands, pushing enter after each (also use ‘man’ to find out what they are doing):

$ echo foo bar baz
$ cat /var/log/system.log
$ echo foo foo foo
$ perl -wle 'print q/hello world/'
$ awk -F: '{print$1}' /etc/passwd

Now we want to execute the first command again, ‘echo foo bar baz’. To do this press ‘Ctrl + r’, the command prompt will change to a semi-colon and the text ‘reverse-i-search’ should be there somewhere as well. Now type ‘echo’, and the most recent ‘echo’ command should be displayed. This is not the one we want, we want the one we typed first, so hit ‘Ctrl + r’ again and now we should see the first echo command. Press the enter key and the command should execute and ‘foo bar baz’ should be echoed to the shell. We could have entered ‘baz’ at the semi-colon and we would have gone straight to the first ‘echo’ command (ie ‘baz’ only appears in the first ‘echo’ command) but now you know how to scroll through the search results as well.

What if we wanted to execute ‘echo foo bar bar’? This differs from ‘echo foo bar baz’ by only a single letter, so we could type out the whole command again, but it might be easier to get the old command and edit it. Lets try doing this, enter ‘Ctrl + r’ and type ‘baz’ at the semi-colon. The command we want should now be there, but instead of pressing enter (which will execute the command), press ‘Ctrl + e’ (go to the end of line). The command we want will now be at the prompt, our cursor will be at the end of the line and we can edit the command. Delete the last ‘z’, hit the ‘r’ key and press enter. We have now executed ‘echo foo bar bar’ with a lot less effort than if we had typed the whole command out by hand.

Revisionism – Event and word designators

For the motivated lazy programmer there is an enormous amount you can do with the event and word designators. I’m an only going to touch on this topic briefly, more to let you know they exist than anything else, but these designators give you access to and allow you to alter previously run commands and their arguments. Roughly speaking event designators give you access to previous commands and word designators allow you to manipulate previous arguments to the commands.

Lets start with the event designators. Try the following:

$ echo foo bar baz
foo bar baz
$ !!
foo bar baz

Thats right ‘!!’ repeated the previous command. From the previous section we now know that we can do the same thing with ‘Ctrl + p’, the up arrow key or by doing a ‘Ctrl + r’ search, so what is the point of another way of doing the same thing? Well this is just the tip of the iceberg. What if you want to repeat a command that you issued ten commands ago? You can use this notation to refer to specific commands in your history. One way you can do this is to use the ‘!-N’ command where N is a number referring to the Nth last command. So to re-execute the third last command try the following:

$ echo one
one
$ echo two
two
$ echo three
three
$ !-3
echo one
one

In this case the ‘!-3’ command resulted in the execution of ‘echo one’ and so ‘one’ was printed. Notice that the command that was executed was also echoed to the screen (ie. the ‘echo one’ before ‘one’ after !-1) just to let you know what you did and make sure you remembered correctly how far back you issued the command. You probably don’t want to use this command for commands in the distant past, it is unlikely that you will remember correctly how far back the command was executed and issuing random commands can result in some unusual behaviour. A safer way to use the ‘!’ notation is to use it in conjunction with the ‘history’ command. Remember that ‘history’ will list your old commands and maybe you noticed that the commands were numbered in the output. We’ll try the history command again to refresh our memories. Type the following and, as always, press enter:

$ history 5
627  echo two
628  echo three
629  echo one
630  history
631  history 5
$ !-628
echo three
three
$

For this example we used ‘history’ with an integer argument (5) and so we got our last five commands echoed to the shell. Each command in our history is shown on its own line along with a number representing the count of commands that we have issued in our session. On my system therefore ‘echo two’ was the 627th command issued in my session. We can now access these commands using the event designator, ‘!’, in the following way:

$ !627
echo two
two

By adding the index number (ie the count of the command) to the ‘!’ we referenced the 627th command in our history. In my case this was ‘echo two’ so this was executed and ‘two’ was printed to the shell (once again the command re-executed was also echoed for your information). What if you remember the command you executed some time ago and you want to execute it again. In this case ‘!’ also comes to the rescue as you can also use the ‘![string]’ notation. If you are following this lesson through you should have the ‘perl -wle ‘print q/hello world/” that we issued in the last section somewhere in your history. To re-execute this command using the ‘![string]’ notation try the following (be careful though, if you have issued any other commands starting with ‘perl’ in the interim, the most recent of these commands will be executed instead):

$ !perl
perl -wle 'print q/hello world/'
hello world

Viola, the last command that began with ‘perl’ was located, re-executed and ‘hello world’ echoed to the shell. Any matching string will work, for example now that we have re-executed this command, ‘!p’ will result in the same behaviour (try it). We can also use this for other parts of the command other than the beginning, the command ‘!?[string]?’ will match any part of a command line in your history and execute the most recent command containing [string]. Try it, lets re-execute the command ‘echo foo bar baz’ by matching against ‘baz’:

$ !?baz?
echo foo bar baz
foo bar baz

Event designators become very powerful when they are combined with word designators. The word designators give you access to the arguments of previous commands and, also, the tools to change what was previously executed. I am not going to say much about these, firstly, because I don’t want to scare you with the arcana of command line intricacies at this stage and, secondly, because it is a hard to conceptualise what you would use these commands for until you know more about the command line and have done enough command line work to find yourself in a position where you are thinking something like ‘It would really be good if I could …’. If you do end up working on the command line a lot I hope you remember this lesson and know where to look for the commands you want. If you do want to learn more a good place to start is the Bash manual and the sections on Event and Word designators.

What I will do is just give an example of how, when used in conjunction, the designators can do some pretty useful stuff. In the last section we did an exercise where we searched our history for the command ‘echo foo bar baz’ and then edited it to execute ‘echo foo bar bar’. We did this using the reverse search (‘Ctrl + r’) and the editing shortcut ‘Ctrl + e’ (go to the end of the line). Using designators we can achieve this in one line of text. If you still have ‘echo foo bar baz’ in your history try issuing the following command and see if you can work out what is happening:

$ !?baz?:s/baz/bar/
echo foo bar bar
foo bar bar

Beautiful, elegant and a boon to lazy programmers everywhere. A little bit of reading, learning and practicing can give you tools to really get some work done while still leaving time for the essentials in life, such as tweeting, drinking and watching TV.

Aliasing and (a little bit of) scripts

We are going to end this post on making things easier at the command line by having a look at the way we can save commands for later use. We can do this in two ways, by setting up an alias or, for more complex tasks, writing a shell script. I will only mention shell scripts briefly as we are here to learn Python programming, but it is worth knowing that the Bash shell incorporates a complete programming language that can be used to write programs as complex and complicated as any other programming language. You probably don’t want to write really complex applications with Bash but for small to medium size programming jobs it can be just the thing you are looking for. Particularly when dealing with tasks intricately entwined with the operating system. In fact shell scripts are the glue that hold many Unix systems together and, as Unix systems are the backbone of the internet, you could say the internet runs on shell scripts (although Perl also makes a claim to this distinction).

Aliases, on the other hand, are a simpler way of providing a shortcut to a common but reasonably lengthy single line command. By setting up an alias you will be setting up something that you will call just like any other command — if you set up an alias called `jklik’ you would type this, just like you would something like `ls’, on the command line and the command aliased by `jklik’ will execute. For example, just say you have a server somewhere that you log into from your home computer using the secure shell program (ssh) using public key encryption. The command that you would issue at the command line would be something like the following:

$ ssh -i /Users/jason/.ssh/somekey jason@some.server

This isn’t a particularly long command, but it can become tiresome if you are often logging into your server. Wouldn’t it be easier to just type a command like ‘someserver’, ‘server or even just ‘ss’ rather than the longer version? I would say yes, and most people would agree. To set something like this we would use the command ‘alias’. Lets set up an alias. We keep issuing commands like ‘echo foo bar baz’, lets set up an alias for this where we will issue the command ‘fbb’. The first thing we should do is check that there is not a command called ‘fbb’ already. Bash will let us call our alias anything we want and if it is the same as an existing command then it will ‘overwrite’ the previous command (you can remove the alias using ‘unalias’ if you do this by accident). For example if we were to call our alias ‘ls’ then we would no longer be able to list the contents of our directories. So to test it the easiest way is to just type ‘fbb’ and press enter:

$ fbb
-bash: fbb: command not found

When we do this we get an error from bash informing us that the `fbb’ command is not found, meaning there is no command by that name and so we can go ahead and use it as our alias. To set up the alias all we need to do is issue the ‘alias’ command with an argument comprised of our alias name, an equal sign and the quoted command we want to alias. In our case it will look like the following. Note that we cannot have a space on either side of the equals sign as the shell will then interpret ‘fbb’ ‘=’ and ‘echo foo bar baz’ as three separate arguments instead of the one argument equating our alias name to the command:

$ alias fbb='echo foo bar baz'
$

If it works then nothing will happen and the Bash prompt will be returned, in this case no news is good news and hopefully our alias command has worked. To test it try issuing ‘fbb’ as a command. You should see ‘foo bar baz’ echoed to the shell — success! What we now have is effectively a new command and command completion and all the other things we have discussed will work on this command just like all the built-in commands. There is a down side though to what we have just done. If we log out of the shell and then come back later our alias will be gone. A transient alias is not much good if we have to set it up every time we log in. There is, of course, a solution to this conundrum but before we can do this we need to learns some more basics. Things like looking in files, editing files and saving our changes. This will all be covered in the next post and hopefully I’ll remember to revisit this issue after we have covered all that. For now hopefully we have made our selves more comfortable on the command line and provided ourselves with some of the basic skills that will make our further work easier and more efficient.

Exercises

  1. Make a cheat sheet which sets out all the shortcuts and techniques covered in this post. Print it out and stick it on your monitor (on the side of the monitor obviously).
  2. Extra, extra, credit: Work out how to make your ‘fbb’ alias permanent. You will need to know about the .bashrc and .bash_profile files, the difference between a login and a non-login shell and the way in which your system treats them in the terminal (for a hint see here).

Comments

comments

Powered by Facebook Comments

Comments are closed.