Automatic Documentation Publishing with GitHub and TravisCI

 Note: This article was written before TravisCI had built-in support for deploying to GitHub Pages. Now that they do, it is recommended to use their GitHub Pages Deployment Guide.

 I love GitHub Pages; they are a great place to put product documentation. I also love Doxygen; it is a great way to generate documentation. I also love Travis CI; it is a wonderful tool for continuous integration. Finally, I love making automated things work together.

At the end of this post, you will have:

• A project on GitHub...
• ...with up-to-date documentation hosted on GitHub pages...
• ...automatically generated and published by Travis CI.

I will assume you have at least a basic understanding of GitHub, Doxygen and TravisCI. I also happen to believe that GNU Make is the greatest build system in the world, so you will have to live with looking at a tiny bit of Makefile.

Create a Project Page

The first thing you need to do is create a GitHub project page. If you already have one, jump to the next section.

The GitHub guide for creating a project page is quick and easy to follow, so I would highly recommend going through it. First-time page creators might worry about issuing the rm -rf . command to their beautiful repo, but don't worry about it! When you said --orphan, you created a branch which is unrelated to your master, so you do not have to worry about deleting your real files. As a second note, it usually takes quite some time for your pages to propagate out to GitHub's CDN, so do not worry if you don't see your changes immediately.

Create an SSH key for Travis CI

Next, we need to create a new SSH key and upload it to GitHub. Do not use your normal SSH key! In a bit, we are going to be putting this new key onto Travis CI's servers. If their system is compromised, you will want to delete access to this SSH key. This is mostly the same as the GitHub guide. Do not use a passphrase for the key -- we are going to use this key from Travis CI, which will not be interactive.

Go to the directory which contains your repository and generate an RSA key pair:

$> ssh-keygen -t rsa -C "youremail@example.com" -f config/travisci_rsa
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in config/travisci_rsa.
Your public key has been saved in config/travisci_rsa.pub.
The key fingerprint is:
ea:31:00:2a:db:9d:6b:c2:68:34:3d:b9:86:18:77:66 youremail@example.com

This will create two files: config/travisci_rsa, which contains the private key and config/travisci_rsa.pub, which is the public one. Upload the public one to GitHub. At the time of writing, there is no way to restrict the use of SSH keys to specific repositories. If that feature is added in the future, it would be a good idea to only allow the key you just generated to access the repository you are working on.

Now we face a minor security problem: We would like to get our private key (config/travisci_rsa) to Travis CI without anyone snooping it. Obviously, we can not just upload it to our public repo. Luckily, the good folks at Travis CI have thought of a great solution to this problem with their ability to encrypt files.

The Travis CI CLI is available as a Ruby Gem. Install it and log in to your repository.

$> sudo gem install travis
$> travis login

Next, we can use the CLI to encrypt the file:

$> travis encrypt-file config/travisci_rsa --add
encrypting config/travisci_rsa for tgockel/nginxconfig
storing result as travisci_rsa.enc
storing secure env variables for decryption
Make sure to add travisci_rsa.enc to the git repository.
Make sure not to add config/travisci_rsa to the git repository.
Commit all changes to your .travis.yml.

This creates a file encrypted with AES 256 CBC with the secret key stored on Travis CI servers. The key will be given to the build script and will only work on builds for your repository. At this point, it is a good idea to delete the unencrypted config/travis_rsa so you do not accidentally commit it. The --add option made the CLI edit your .travis.yml file to include a step for decrypting the file in the before_install section of your build.

For some reason, the CLI puts the encrypted file in the root directory, even when the original file is somewhere else, so move it back to the config directory with a mv travisci_rsa.enc config. We also need to edit the .travis.yml to point to the correct location. Make your entry look something like this:

before_install:
- openssl aes-256-cbc
  -K $encrypted_dc9ab2986818_key
  -iv $encrypted_dc9ab2986818_iv
  -in  config/travisci_rsa.enc
  -out config/travisci_rsa
  -d

To use the key on the server, we need to add it to our SSH config. We also need to set the key file to only be readable by this user or SSH will refuse to use the key. So, add these two steps after the openssl decryption step:

- chmod 0600 config/travisci_rsa
- cp config/travisci_rsa ~/.ssh/id_rsa

Publishing Documentation

We've got all the components ready for Travis CI to put things into our git repo, so let's have it do so! First, let's get Doxygen generating documentation. If you do not have your project building documentation yet, create an initial configuration file with:

$> doxygen -g config/Doxyfile

This will generate a generic configuration file for generating documentation. Use a text editor to edit all the options to your liking (there are a lot of them, but the documentation is awesome). I will assume the following options have been set:

OUTPUT_DIRECTORY       = build/doc
GENERATE_HTML          = YES
HTML_OUTPUT            = html
HTML_TIMESTAMP         = NO

If the output directories are different, then you can modify the script to point at different locations. I prefer setting HTML_TIMESTAMP to NO so my documentation repository updates more closely resemble my actual repository updates, but that is not a big deal. If GENERATE_HTML is NO, then the HTML documentation will not be generated, making this how to less useful. Test that your Doxyfile will generate the desired output with:

$> doxygen config/Doxyfile

Open up build/doc/html/index.html to check your settings. If you do not like what it looks like, tweak the Doxyfile until you are happy. This will be what the result looks like.

I prefer having all of my build outputs in my Makefile, so I added this recipe to mine:

doxygen :
    @echo " DOXY  nginxconfig"
    @doxygen config/Doxyfile

Travis CI will happily execute any script you give it, so we need to write a script that will generate the documentation into the gh-pages branch of our repository. So let's create a bash script in config/publish-doxygen (for the impatient, the end result is here).

First, I like putting all of my settings at the top of things so they are nicely grouped.

REPO_PATH=git@github.com:tgockel/nginxconfig.git
HTML_PATH=build/doc/html
COMMIT_USER="Documentation Builder"
COMMIT_EMAIL="travis@gockelhut.com"
CHANGESET=$(git rev-parse --verify HEAD)

Most of these are fairly self-explanitory. Update REPO_PATH to reflect your actual repository. Likewise, change COMMIT_USER and COMMIT_EMAIL to something meaningful to you (unless you want me to build all of your documentation). The only funny thing is CHANGESET, which will be the current changeset of the repository. I will get back to it later.

First things first, let's get a clean version of the HTML documentation repo:

rm -rf ${HTML_PATH}
mkdir -p ${HTML_PATH}
git clone -b gh-pages "${REPO_PATH}" --single-branch ${HTML_PATH}

This is just checking out the existing gh-pages documentation into where we are going to be generating it.

The next step is to clean up the existing documentation by completely deleting it:

cd ${HTML_PATH}
git rm -rf .
cd -

WHOA! Why delete everything? It turns out, git is pretty smart about detecting changes in files. The next step is to regenerate the documentation, so git will detect that certain files came back. If you do not do this step, then you can end up with a bunch of stale, unused files.

Assuming you created a step in your Makefile, the next bit is simple:

make doxygen

Now we have the HTML documentation generated into build/doc/html, so we just have to get it published via the gh-pages branch.

cd ${HTML_PATH}
git add .
git config user.name "${COMMIT_USER}"
git config user.email "${COMMIT_EMAIL}"
git commit -m "Automated documentation build for changeset ${CHANGESET}."
git push origin gh-pages
cd -

This is all pretty sandard-fare git configuration. Remember -- a Travis CI build VM comes to you completely fresh. The only slightly interesting thing here is the use of the CHANGESET variable. Dropping this into the commit message makes GitHub link to the changeset in the main repository (example), which I find very convenient. I would recomment that you test this script out locally before committing it.

The final step is to tell Travis CI to actuall run our publish-doxygen script. Unfortunately, Doxygen and some of the components that make Doxygen useful are not installed by default, so they need to be added to your .travis.yml.

install:
- sudo apt-get install --yes doxygen graphviz

I like putting installation of libraries under the install segment, but it is not really required. This can make the installation step take a very long time, since it involves downloading a ton of LaTeX-PDF tools. You might consider putting it into your after_success step if you want your actual product build to happen sooner.

Finally, we need to execute the script. You probably only want to execute if the build was successful, so the after_success section of .travis.yml is a logical place to put this:

after_success:
- ./config/publish-doxygen

That's it! Commit that and upload it to make Travis CI automatically generate and deploy documentation with every successful build. Awesome!