Cloud Build Recipes - Incrementing Build Numbers

Many CI services use an incrementing build number to identify builds. GCB, however, uses a randomly assigned unique identifier, as is common in large scale systems. While this is common and helps make GCB infinitely scalable, it makes it difficult to reason about the relationship between builds.

Since I like to use the CI build number in my Android app versioning, I decided to reimplement this functionality using a simple script and text file stored in a shared location. The idea is simple, each build mounts a shared config folder, reads the version number from a text file, and overwrites that text file with a new version number.

In a highly concurrent build environment there is a chance of collisions, resulting in multiple builds with the same build number, but for my small team and limited builds, this isn’t likely.

The .buildenv Script⌗

The script that I use to increment the build number, in addition to overwriting the original file with the new build number, actually creates a very simple bash script locally that can be used to set a BUILD_NUM environment variable in future build steps.

#!/bin/bash

export CI=true
export BUILD_NUM=43

The reason that I create a bash script for this is in GCB, all build steps are isolated from each other, only sharing the file system. This means that environment variables set in one step won’t be carried to the following steps.

A simple solution is to wrote a script to the file system and let future steps source the script when they start up. In my environment, I’ve taken to calling this script .buildenv.

source .buildenv

Deploying the Docker Container⌗

To save you from having to create your own docker container and script to manage your build numbers, I’ve created a buildnum container in the Android Cloud Build repository.

There are instructions for deploying the container to your own Google Cloud Registry (GCR) for use with GCB in the readme, but as a quick summary, the container can be built using the following GCB command, which will automatically upload it to GCR for you.

gcloud builds submit --config=cloudbuild.yaml

Configuring the Build⌗

The three steps mentioned above, mounting the config folder, getting and incrementing the build number, and saving the updated config file, need to be run sequentially at the beginning of our build process.

The first step, copy_config, uses the gsutil command to copy the contents of the Google Cloud Storage (GCS) bucket to a local shared volume, called config.

- name: 'gcr.io/cloud-builders/gsutil'
  id: copy_config
  waitFor: ['-']
  args: ['rsync', 'gs://${_CONFIG_BUCKET}/', '/config']
  volumes:
  - name: 'config'
    path: '/config'

It’s important to use rsync instead of cp here, as cp will fail if no files exist, which will be the case the first time the build is run.

Also note the waitFor option in this build step. Setting the value to - means that this build step will be run when the build starts, without waiting for any other build steps.

The next step is to run the buildnum script to actually generate and update the build number. This step uses the buildnum container that you previously deployed to your Google Cloud Registry.

- name: 'gcr.io/$PROJECT_ID/buildnum'
  id: setup_env
  args: ['/config/buildnum', '.buildenv']
  waitFor: ['copy_config']
  volumes:
  - name: 'config'
    path: '/config'

The buildnum script takes two arguments, the source version file (/config/buildnum), and the destination script (.buildenv). This adds flexibility, allowing you to use different version number tracking for different builds.

In this step we also have to set the waitFor property so that this step is run only after the copy_config step completes.

For the final step, we copy the updated /config/buildnum file back to our config GCS bucket for future builds.

- name: 'gcr.io/cloud-builders/gsutil'
  id: save_config
  args: ['cp', '/config/buildnum', 'gs://${_CONFIG_BUCKET}/buildnum']
  waitFor: ['setup_env']
  volumes:
  - name: 'config'
    path: '/config'

This step waits for the setup_config step to complete so that we have access to the updates buildnum file.

The Graph⌗

To help visualize how these steps fit together in our build, here’s a graph of the build steps. While this is super simple, notice how our general build steps only wait for the setup_env step to complete.

This allows the save_config step, which simply copies the updated config file back to GCB for future builds, to run in parallel with the rest of the build steps and makes the entire build faster.

That’s All⌗

Stay tuned for more recipes to help make Google Cloud Build a really great continuous integration service for your Android apps.