Linux Command Works in Shell but not in Crontab

April 9th 2022/Share/Edit

This guide will walk you through some common woes to running cronjobs successfully.

So, if you have a command working normally in your regular user environment and then copy and paste it into crontab, and nothing happens. This guide's for you.

After a month of on and off investigations, here's the steps that helped:

Exporting your PATH + SHELL variables

Cronjobs run in a very limited environment compared to your regular user.

So, if you're copy and pasting your command straight into crontab, then chances are this issue will be cropping up for you.

Open your cron file via crontab -e and set your PATH and SHELL variables before your actual cronjobs.

#
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
#
# m h  dom mon dow   command
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

You can find these values by logging them via the echo cmd

echo $PATH
echo $SHELL

Escape Percentage Symbols

Percentage symbols in cronjobs are used to indicate new lines.

so, if you're using them in a command, you have to escape them.

man (5) crontab:

Percent-signs (%) in the command, unless escaped with backslash (\), 
will be changed into newline characters, and all data after the 
first % will be sent to the command as standard input.

This also applies whether the percentage signs are wrapped within quotes or not.

# unescaped command
youtube-dl -o "/vids/%(uploader)/etc..."
# escaped command
youtube-dl -o "/vids/\%(uploader)/etc..."

PGREP: Conditionally run Commands

You sometimes might want to automate a task on the condition that it isn't already running.

you can accomplish this via the use of pgrep

pgrep youtube-dl
# output:
# 1178150

pgrep takes the commands name, and lists all the processes that are running it.

If no processes are running, then it exits with a non zero exit code.

Which means we can chain it with an or condition, and send the stdout to /dev/null

pgrep your-command > /dev/null || your-cron-job-cmd