Please wait until that HTTP service is ready

Our brave new world of containers and microservices runs on a combination of YAML and shell scripts.
Getting them to run in the desired sequence can be a challenge.

When ready isn't ready

All container environments have a "depends" clause, so the container runtime can determine the correct startup sequence for the zoo of containers comprising the application to be launched. Each container will usually signal when it is ready.

However ready can mean different things to different applications.

In the container world it should be: the service is available.
However it could be: service was successfully started, but might be busy with house keeping. In the later case the start scripts of the dependent services need to do their own waiting

curl to the rescue

Curl is the Swiss Army Knive of network tools. It probably takes a lifetime to understand all its possibilities. Make sure you have the current version, but at least 7.52.0 from 2016.

Whipping up a little shell script provides what we need:

#!/bin/bash
# Wait for a http service to be available
export TARGET=http://baseservice.local/
curl -v \
     --retry 10 \
     --retry-delay 20 \
     --retry-connrefused \
     $TARGET

if [ "$?" -ne 0 ]; then
  echo "Failed to reach $TARGET"
  exit 1
else
  echo "$TARGET has replied!"
fi

Let's break it down:

• export TARGET defines the URL you want to reach. Works for both http and https. Potentially works for other protocols supported by curl as well, but I haven't tested that (yet). You might swap it out for $1 to capture the first parameter called
• \ denominates line continuation. The whole command needs to be on one line which is hard to read. \ tells Linux to treat the following line as belonging to the current one. On Windows you need to replace \ with ^ and alter the syntax according to Powershell
• curl -v Verbose output, so the logs tell the full story. When you work with privat CA or self signed certificates (Don't do self signed, use your own CA), you need to add -k or --insecure to make this work
• --retry 10 try the connection 10 times
• --retry-delay 20 Wait 20 sec until the next try
• --retry-connrefused retry even on "connection refused" responses. By default curl retries only http responses not in the 2xx range. Connection refused is HTTP's way telling you: there's nothing taking your call, which curl, without --retry-connrefused takes as final, fatal error and terminates retries. When you have curl v7.71.0 (from 2020 or later), you can use --retry-all-errors instead. The documentation warns This option is the "sledgehammer" of retrying. Use it wisely
• if [ "$?" -ne 0 ]; then Commands usually return a numeric result. Anything other than 0 indicates a failure condition of some sort. When curl has exhausted the number of retries a value greater 0 will be returned. To capture the return value we us $?
• return 1 We indicate that our script failed. So it can be called from other scripts

We could swap out $TARGET for $1 which makes the script callable handing over the target URL e.g waitforme.sh http://someURL

Beyond containers

The script is also useful in "classic" situations, where you start a service in the background (which by definition is asynchonous). Your local environment uses anything like:

• ./startdb &
• systemctl start myservice
• domino[_container] start

You can use the curl retry for any automation. Adjust the wait time and the retry time to your needs.

As usual YMMV