Day 4: Scripting Docker Commands With Spinup.sh

In Day 3, I included a blurb from my DevOps-y friends about the natural progression of abstractions on top of containers:

You usually start with docker run CLI commands and graduate to tools with more layers of abstraction as you need them. Docker-compose comes next, followed by automating several commands with Bash scripts, which is eventually followed by Kubernetes.

I also shared with you a better way to handle switches in Bash scripts. Today I’ll show you how I moved from running my own Docker commands to running off of one shell script with a handful of flags.


For the first few months of learning how to use Docker, and then how to utilize it in projects, I was running A LOT of individual Docker CLI commands every day. I then moved on to having Docker Compose manage things for me. However, even that started to be a burden for some edge cases I was dealing with.

After thinking back on my conversations with other friends in the DevOps field, I remembered they all told me that you’ll hit a certain point of wanting to automate your workflows… so I started diving into Bash scripts.

I knew I would want my script to do a few different things:

  • Spin up all of my container infrastructure
    • One version with explicit commands for each piece of infrastructure (dev)
    • One version running off docker-compose (prod)
  • Tear down all of my container infrastructure (teardown)

First thing I did was to make sure my docker-compose.yaml file was built out. That would be my source of truth – if running docker-compose up -d made everything work correctly, then the rest of this script would be based on what was written in that file.

version: "3.7"
services:
app:
container_name: hquinn-app
image: "hquinn_app:latest"
networks:
- hquinn-net
ports:
- "8080:8080"
restart: always
volumes:
- type: volume
source: hquinn_app_home
target: /var/www/html
networks:
hquinn-net:
volumes:
hquinn_app_home:

I felt that I should point out that this is just an example project. hquinn-app isn’t a real image, container, or project. Use this as a template to plug in your own information. It’s a good training exercise!

Running docker-compose up -d seems to work with this yaml configuration. Good! Now we can move on to creating our Bash script. We’re going to call this spinup.sh.

Let’s start by setting the improved switch statement that we learned about yesterday. We’ll include four flags (help, dev, prod, teardown) as well as a catchall for errors.

#!/bin/bash
while getopts ":hdpt" opt; do
case${opt}in
h )
printf"USAGE: ./spinup.sh [OPTION]... \n\n"
printf"-h for HELP, -d for DEV, -p for PROD, or -t for TEARDOWN \n\n"
exit 1
;;
d )
exit 1
;;
p )
exit 1
;;
t )
exit 1
;;
\?)
echo"Invalid option: %s""$OPTARG"1>&2
exit 1
;;
esac
done
shift $((OPTIND -1))
printf"USAGE: ./spinup.sh [OPTION]... \n\n"
printf"-h for HELP, -d for DEV, -p for PROD, or -t for TEARDOWN \n\n"
exit 1
;;

Solid. This switch is going to make it really easy to just plug in commands we want to run for each flag.

Production will probably be the easiest since we’ll be leaning on the docker-compose.yaml files we already built out. Let’s fill those commands into the p ) case:

p )
# Rebuild image
docker-compose build
# Spin up container
docker-compose up -d
exit 1
;;

As you can see, we’re really just having this bash script run the same commands we would run ourselves to start up our containers, volumes, and networks. We’re just splitting up the different jobs into different flags so we can utilize the same script to accomplish a number of different tasks.

Our dev case [d )]isn’t going to be much different. We’re just manually creating a network and running one long docker run command to start up our container:

d )
# Rebuild image hquinn_app
docker-compose build --no-cache
# Create hquinn-net bridge network for container(s) to communicate
docker network create --driver bridge hquinn-net
# Spin up hquinn-app container
docker run -d --name hquinn-app --restart always -p 8080:80 -v hquinn_app_home:/var/www/html --network hquinn-net hquinn_app:latest
exit 1
;;

Henry, what’s the actual difference between your dev and prod builds here?

Great question, reader! This is part of the fun (pain?) of initially learning about containers. There are a lot of different ways of dealing with the same tasks and you learn best practices as you go.

When I initially wrote this script, I was working on that ColdFusion, Informix, and MySQL project. Due to the way it was initially built before it was handed to me, we needed to run different sets of commands to spin it up depending on if we were running it locally for development or if we were running it in production for actual use by judges.

As I dug deeper into Docker, I had all kinds of sources telling me what should have been obvious:

One of the main tenants of containers is that your code should run the same everywhere. It’s the same containers, just running on different engines.

That’s to say that I should be running the same commands to run the same containers everywhere. Since I wasn’t, I was still falling prey to the whole but, it worked on MY machine gotcha.

Since then I’ve trimmed this script down a bit. I still like having the longer commands in my d ) case, though. It allows me to test some things quickly in the way that I stand up my infrastructure that I can then solidify in my docker-compose.yaml files that I can then run in production environments. This is another tenant of containers, we can treat our infrastructure as code. Once our docker-compose.yaml is fine-tuned to our liking, we can check it into version control and know that it’s safe for all time.

Now the t ) case is meant to tear down all of our infrastructure. Kill containers, and remove containers, images, volumes, and networks. That way we can get a clean slate to spin up and test out new changes we made to our infrastructure.

We’re going to accomplish this with a number of if/then blocks:

# If hquinn-app container is running, turn it off.
running_app_container=`docker ps | grep hquinn-app | wc -l`
if [ $running_app_container -gt "0" ]
then
docker kill hquinn-app
fi

For this particular block, we’re setting a variable named running_app_container to the output of docker ps | grep hquinn-app | wc -l. Which means if the container hquinn-app is up and running, running_app_container is set to the number of lines returned by that command.

The if/then block then checks to make sure the controlling variable is greater than 0. If true, it runs the command docker kill hquinn-app to kill the container.

We’ll use a series of these blocks to manage our containers, images, volumes, and networks.

Let’s see the entire spinup.sh script, with all of the parts plugged in:

#!/bin/bash
while getopts ":hdpt" opt; do
case${opt}in
h )
printf"USAGE: ./spinup.sh [OPTION]... \n\n"
printf"-h for HELP, -d for DEV, -p for PROD, or -t for TEARDOWN \n\n"
exit 1
;;
d )
# Rebuild image hquinn_app
docker-compose build --no-cache
# Create hquinn-net bridge network for container(s) to communicate
docker network create --driver bridge hquinn-net
# Spin up hquinn-app container
docker run -d --name hquinn-app --restart always -p 8080:80 -v hquinn_app_home:/var/www/html --network hquinn-net hquinn_app:latest
exit 1
;;
p )
# Rebuild image
docker-compose build
# Spin up container
docker-compose up -d
exit 1
;;
t )
# If hquinn-app container is running, turn it off.
running_app_container=`docker ps | grep hquinn-app | wc -l`
if[$running_app_container -gt"0"]
then
docker kill hquinn-app
fi
# If turned off hquinn-app container exists, remove it.
existing_app_container=`docker ps -a | grep hquinn-app | grep Exit | wc -l`
if[$existing_app_container -gt"0"]
then
docker rm hquinn-app
fi
# If image for hquinn_app exists, remove it.
existing_app_image=`docker images | grep hquinn_app | wc -l`
if[$existing_app_image -gt"0"]
then
docker rmi hquinn_app
fi
# If hquinn_app_home volume exists, remove it.
existing_app_volume=`docker volume ls | grep hquinn_app_home | wc -l`
if[$existing_app_volume -gt"0"]
then
docker volume rm hquinn_app_home
fi
# If hquinn-net network exists, remove it.
existing_hquinnnet_network=`docker network ls | grep hquinn-net | wc -l`
if[$existing_hquinnnet_network -gt"0"]
then
docker network rm hquinn-net
fi
exit 1
;;
\?)
printf"Invalid option: %s""$OPTARG"1>&2
exit 1
;;
esac
done
shift $((OPTIND -1))
printf"USAGE: ./spinup.sh [OPTION]... \n\n"
printf"-h for HELP, -d for DEV, -p for PROD, or -t for TEARDOWN \n\n"
exit 1
;;

All in all, this is looking pretty tight. You can add more commands in if you need anything more complicated. You can add more flags to handle more edge cases, too.

This script (and a handful of others like it) really helped me through the last six months of my job with the courts. However, with the projects I’m working on now, the amount of these scripts I would need to remain productive is going to be a burden to maintain. We need more power and more control over what we’re doing.

Hence, my deep dive into Kubernetes.

I haven’t forgotten about it. I’m starting to dig into the books that I bought. As far as Kubernetes THe Hard Way goes, Christian Corbin pointed out that the tutorial might be out of date. To that end, I think I’m going to drop K8s The Hard Way and just focus on the books that I bought and the Kubernetes.io when I need some hands-on practice.

DevOps is all about iterating on and improving processes. Happy to change things here as better opportunities come up!


It’s a holiday weekend and I’m headed to Maine. Time to spinDOWN.sh.

/rimshot

I’ll try to write some more while I’m on vacation, though you might not hear from me until next week.

Stay frosty.

https://henryneeds.coffee
Blog
LinkedIn
Twitter

Day 3: Fun With Flags

First off, some housekeeping. I can already tell that coming up with a daily post is going to be harder than my buddy Alex had with his SVG work. He had one animation to complete every day. I’m spending all month working towards one larger goal.

However, I’ll do my best to share something I learn every day, even if it’s small.


Background

I bought Nigel Poulton’s book bundle from LeanPub and am starting to dig into those, but learned something neat recently that I thought I would share.

When I was still at the courts, I was teaching myself how to use containers as quickly as I was implementing them in projects. While learning how to manage multiple sets of containers, I spoke with my college friends who were in DevOps-y roles about the tools they used. It shocked me to hear everyone say the same thing:
Continue reading Day 3: Fun With Flags

Day 2: What’s the plan?

In the spirit of learning in public, I wanted to share why I picked Kubernetes as the topic of my deep dive, what my defined goals are for the end of the month, and what my learning plan looks like in order to get me over the finish line.
As short as a month can feel, this is a marathon; not a sprint. Let’s take these in order:
1. Why did I pick K8s?
I have a bit of a weird background: computer nerd in a high school with no computer courses -> Champlain College CNIS student -> DBA at the US Courts -> web developer at the US Courts -> web and infrastructure developer for the AO of the US Courts -> DevOps Engineer at Clarity Software Solutions.

Continue reading Day 2: What’s the plan?

Day 1: Learning DevOps In Public

Hi, my name is Henry Quinn and in July I’ll be “learning in public.”

Last month, my buddy Alex Trost started a learn in public month after reading this post by Shawn Wang. In 30 short days he really upped his game with SVG animations (enough where he was asked to create an eLearning course about it), so I figure there must actually be some benefit to learning, and then teaching, something new every day.

Continue reading Day 1: Learning DevOps In Public

Deepfakes PSA

Looks like it’s time for another PSA, y’all. Buckle in.

Unless you’ve been living under a rock since the 2012 U.S. presidential election cycle, you know that “fake news” has been running rampant. We’re in a real life age of Nineteen Eighty-Four-esque doublespeak fueled by literal White House Press Secretaries and anyone savvy enough to set up a blog. The very offices and organizations that are supposed to tell us where we stand on the world stage are lying in very public forums about easily verifiable facts.

Shit’s real and shit’s scary.

Here’s the real rub, though: it’s about to get a LOT worse.

Continue reading Deepfakes PSA

From Barely Functional PHP To Conference Worthy Web Apps In Under Two Years

I’ve only been in the web development game for a little under two years, but I’m building applications for a federal agency that are getting me invited to speak at conferences. It took me a while to figure out how to get to that point, and I didn’t have anyone available to give me a road map when I started. I spent a long time teaching myself, and in retrospect there are definitely easier paths to take. So here’s my attempt to set new developers on the right track.

Continue reading From Barely Functional PHP To Conference Worthy Web Apps In Under Two Years

$(“#Blog”).append(“Post”);

I haven’t been updating this nearly as much as I should have been. Truth be told I’ve been ignoring some of my yearly goals. Guitar has all but fallen to the wayside, and I’m honestly okay with that. It’s one of those things I’d love to be good at, but really don’t want to put the effort into.

Continue reading $(“#Blog”).append(“Post”);

NART 2013 | Planning

Trust your gut, but do what makes for a better story.

None of this was even an idea yet. All I knew is that I was tired of being told by my supervisors in all too many ways that my behavior as a resident advisor had been disappointing. Had they hired me to watch a small dorm full of similarly aged college kids? Sure. Were they paying me money to hang out with residents that I would have been spending time with anyway? Absolutely. I owed them for that, but after being a camp counselor for so many years and developing a rhythm to how I deal with a disturbance, I wanted to do this job my own way.

Rachel set her pen and notebook on her desk before looking me dead in the eye.

Continue reading NART 2013 | Planning