When you manage servers, it can be useful to take a number of actions automatically (restart a service, add a firewall rule, kill a process, send an alert, etc.) if the load average is too high. The problem with many shell scripts for this is that, if the load average drops slowly after the action, the script may go wrong and trigger another kill, restart, etc. That can cause service downtime.
The solution is to check the load average twice before running your commands.
Now for the code:
#!/bin/sh
tsleep=2 # time to wait before 2 checks
llimit=8 # load limit before action
alert=monitor@christophe-casalegno.com #put your monitor alert email here
host=`hostname -f`
load=`cat /proc/loadavg |awk {'print $1'}|cut -d "." -f1` # The load average now
sleep $tsleep
load2=`cat /proc/loadavg |awk {'print $1'}|cut -d "." -f1` # The load average after tsleep
if test "$load" -ge $llimit
then
if test "$load2" -ge $load
then
date=`date`
echo "The Load Average has reached $load1 and $load2 on $host" | mail -s "$host : High Load Average Alert" $alert
echo "$date : The Load Average has reached $load1 and $load2 on $host" >> /var/log/loadavg.log
else
echo "ok" 1>&2
fi
else
sleep 1
fi
Now just put this script in your crontab, every minute for example, and say good night to your server ;). You can also download this script directly from: https://www.christophe-casalegno.com/tools/loadcheck.sh
Christophe Casalegno
You can follow me on: Twitter | Facebook | Linkedin | Telegram | Youtube
1 comment