Making Cluster-SSH (and regular SSH) a lot more usable with regard to reconnecting

If you find yourself patching a lot of machines at once, and reboot, then your SSH window will close.... not very useful if you want to keep track of a number machines you need to log back into to check that all is okay, or to start services that don't start automatically. It makes that time of the month -- patching -- rather more tedious and painful than it ought to be.

Enter a useful tool called Cluster SSH (command name 'cssh', package name 'clusterssh', version used is 3.28 from EPEL). It distributes my keystrokes to all of the windows that it starts. You can toggle, add and remove hosts to manage, and you can configure clusters of machines. While it does lack polish, it is very useful in reducing the amount of time it takes to patch a lot of machines; I estimate that it takes the time required to about a third.

Here's an example of using it 'in anger' while patching 37 machines. I've deliberately made the image small enough so as to make any text on screen unreadable. I'll admit, my workstation is a little ... odd, but it used to have all three monitors side-by-side in portrait mode before today.

Cluster SSH does have a bit of an annoyance though: if you're rebooting then you lose your windows. It would be nicer to have it in each window do something like prompt you with 'Do you wish to reconnect (y|N)?' I thought so, so I configured it to use an existing SSH wrapper I wrote some time ago: sshr (the 'r' is for 'reconnect').

#!/bin/bash

on_sigint()
{
    >&2 echo "^C"
}

trap on_sigint SIGINT

while true
do
    >&2 echo "Attempting to ssh $@"
    ssh "$@"

    ret=$?

    read -p "ssh returned $ret: run again? [y|N] "
    case "$REPLY" in
        y|Y|yes|Yes)
            continue
            ;;
        *)
            break
            ;;
    esac
done

Now it just need Cluster SSH to be configured to use it. Assuming you put the script in /home/YOU/bin/sshr, configure your ~/.csshrc as follows:

...
ssh=/home/YOU/bin/sshr
...

Here's another productivity tip that comes in useful here; if you need to log into machines according to data in a spreadsheet, make use of filters so only the machines of note are listed; then copy the cells that contain the fully-qualified hostnames of the servers you need to log into. Assuming you have VISUAL=vim (or according to your preference), then in a terminal, type Ctrl-X Ctrl-E to edit a command-line in your editor. Paste in the data from the spreadsheet.

foo.alpha.beta.com
bar.alpha.beta.com
bux.alpha.beta.com

Join all of the lines into one, separated by a space (hint: in vi hold down the 'J' key), and put 'sleep 3; cssh ' at the beginning. Save and exit and it will run.

sleep 3; cssh foo.alpha.beta.com bar.alpha.beta.com bux.alpha.beta.com

The 3 second delay allows me to switch over to a different workspace before it gets flooded with terminal windows.

Comments

Popular posts from this blog

ORA-12170: TNS:Connect timeout — resolved

Getting MySQL server to run with SSL

From DNS Packet Capture to analysis in Kibana