How I Learned to Stop Worrying and Love the Shell Script

Lets get real: shell scripting, love it or hate it, but sooner or later you’ll have to deal with it.

In the past few days I had to write some shell scripts to handle the deploy of a project and, after several trials and errors, I thought “shell scripting is hard, I fixed this problem but suddenly another one appears and I didn’t even understand why”. So, after a brief online research I came across some best practices when one has to write or debug a shell script:

1. Use errexit with set -e

#!/bin/bash

set -e

According to the bash man page (supposing we use Bash in this case), if set -e is used, the script will exit immediately if some command, subshell command, or a pipeline exits with a non-zero status. If you like to design fail-fast systems, this is a nice-to-have option. Even so, set -e has its gotchas.

2. Use nounset  with set -u

#!/bin/bash

set -u

The man page for this option says: Write a message to standard error when attempting to expand a variable that is not set, and if the shell is not interactive, exit immediately. With this option, we can avoid catastrophic actions, like shown below:

$ rm -rf $STEAMROOT/*

If the variable $TEAMROOT is unset, the command will be interpreted as:

$ rm -rf /*

3. Use xtrace with set -x

#!/bin/bash

set -x

The man page says: Write each command to standard error (preceded by a ‘+ ’) before it is executed. This option is very handy if you are trying to understand what is a script doing, kind of a debugging. I’m not very knowledgeable about shell scripting so using set -x gives me a confidence layer by printing the script execution steps.

4. Or, use all of the above options together with set -eux

#!/bin/bash

set -eux

5. Lint you script with Shellcheck

Shellcheck is handy tool to have in the developer’s toolbox. Its goal is to search for both typical begginer syntax issues and advanced subtle caveats, among other classical problems. Just take a look to the gallery of bad code to have an idea of these issues.

You can install and start using Shellcheck right now by running:

$ apt-get install shellcheck

on Debian based systems or:

$ brew install shellcheck

for macOS systems. For other OS, check the project’s page instructions. If for some reason you don’t want to o can’t install Shellcheck on your system, Shellcheck’s website has an online editor where you can write or paste your shell code to be linted.

Possible problems with set -u

Not all the shell applications are 100% prepared to run under (ba)sh strict mode. For example, tools like virtualenv (as of v15.1.0) fails when is executed in script with set -u defined. In those cases, it’s possible to temporarily desactivate the option, run the conflicting command and re-enabling the option afterwards, like so:

#!/bin/bash
set -eux

# Run commands

set +o nounset
# The command below fails if set -u is defined
source path/to/env/bin/activate"
set -o nounset