Continuous integration (CI) is fast becoming part of the standard development model for creating high quality software, and rightly so. Delivering reliably high quality software is in the best interest of yourself, your manager and your clients. This is not an article about Continuous Integration itself, but will introduce Hudson, a key tool, and may help you to implement your own CI strategy. Last year we also published a tutorial on phpUnderControl, which is another CI tool that you might like to consider.
Hudson itself is written in Java, however the folks over at hudson-ci.org have made its installation as pain-free as possible. This article will focus on setting up Hudson on a debian-based platform however the general outline will be useful even for use on a different platform; there are packages available for other linux distributions over on the hudson-ci.org website.
At the heart of continuous integration is unit testing and for this we will be using PHPUnit. There are plenty of good articles about how that is done so we will not cover unit testing in general here. If you want to learn more about the concept of testing code then head over to phpunit.de and read through some of the tutorials there. There is no substitute for experience though, so expect to make plenty of mistakes! Remember, any test is better than no test, and you will improve with practice.
Having tests for your code is a great start, however to really get the benefit, you need to run them frequently . This enabled you to identify whether any new features have changed the behaviour of existing tested code. This is even more important when you’re working in a team of developers, all working on the same codebase and potentially causing issues on different areas of the system (I am confident that everyone reading this will be able to relate to such an experience!).
Running the tests regularly and reporting on the outcome is where Hudson comes in. Integrated with your version control system and given some information about your project, Hudson is able to monitor the contents of your repository. When something changes, it pulls any changes and triggers your suite of tests. As well as running the tests and reporting on the outcome, it keeps track of your level of code coverage (the percentage of code that’s actually run by your test suite – ideally, but rarely practically, this will be 100%). Hudson also records the number of successful vs failed builds, considered to be a measure of the stability of your project. Hudson also supports a vast number of plugins that allows you to enable more complex interactions with your build. It is this pluggable architecture that means we are able to use what Hudson provides on our own PHP projects.
This tutorial is based on a fresh installation of debian stable (lenny), however there is no reason why you cannot setup a CI server along side something else if you need to.
Building a Continuous Integration Server with Hudson
Installing PHP
Unit tests are all run on the command line so we only require the PHP CLI package on our server rather than the full Apache/PHP stack. As well as PHP, we also need the PEAR package installer so we can get the correct versions of the software we need and some of the PHP developer tools. Install PHP, PEAR and their dependencies with the following :-
apt-get install php5-common php5-cli php-pear php5-dev
Now that PEAR is installed, you will need to upgrade it to it’s latest version, we use PEAR itself to achieve this.
pear upgrade pear
Installing Xdebug
The PHP developer tools contain all the command line tools required to build a PHP module. We will install Xdebug from PECL, this will automatically be compiled for us. There is a version of Xdebug in debian’s repository, but it is too old to be compatible with the latest version of PHPUnit, so the preferred installation method is to get it from PECL, which we have access to since we have already installed PEAR.
If you have not used Xdebug before, it is an excellent development tool for PHP and I would highly recommend finding out more about it. Its role on a continuous integration server is to produce of the code coverage reports I talked about earlier in the article; it also has advanced debugging capabilities which can be very helpful when identifying the behaviour of code in order to isolate and identify problems.
apt-get install make pecl install xdebug
Once PECL has built the Xdebug module, you need to tell PHP to load it. You can do this by creating the file /etc/php5/conf.d/xdebug.ini with the contents ‘zend_extension=/usr/lib/php5/20060613+lfs/xdebug.so’.
Verify that the installation has worked by running php –version from the command line. You should see something like the following (version numbers may differ)
ben@ci:~$ php --version
PHP 5.2.6-1+lenny9 with Suhosin-Patch 0.9.6.2 (cli) (built: Aug 4 2010 03:25:57)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
with Xdebug v2.1.0, Copyright (c) 2002-2008, by Derick Rethans
Installing PHPUnit
Hopefully you are already familiar with PHPUnit. It isn’t available from the standard PEAR repository, so you will need to add a new channel for it. You can do this by following the installation instructions, copied here for completeness
pear channel-discover pear.phpunit.de pear channel-discover pear.symfony-project.com pear install phpunit/PHPUnit
Installing Java
Hudson requires the Sun version of the Java JRE to operate normally. This is only available in the non-free repositories so you will need to add this to your apt sources file. Edit /etc/apt/sources.list and add ‘non-free’ to the official repositories by adding ‘non-free’ to the end of the first two lines (beginning “apt” and “apt-src”). Once this is done, run the following to install Sun java6.
apt-get update apt-get install sun-java6-jre
Installing Ant
For PHP, ant acts as the glue between your tests and Hudson. Hudson simply issues a command to ant, and expects to find a number of reports within a preset directory once that command has finished its execution with no errors (an error would indicate a failed build – hopefully something that will rarely happen!). I will talk about how to set up ant with hudson later on in the article but for now, we just need to install it.
Installing ant will include a number of dependencies along with it, so be prepared for a short wait while it is all installed.
apt-get install ant
Some readers may wonder why I am installing ant over Phing. If you wish to use Phing as your build tool, feel free to do so; there are plugins available in Hudson to build using Phing too so it is absolutely a valid option here.
Installing Hudson
Before installing Hudson, if you’re installing on a fresh debian OS you will need one more dependency, daemon. Install it as normal using the apt tool.
apt-get install daemon
Now the dependencies are in place, we can move forward and install Hudson itself. Hudson is not available in the main debian repositories – but installation is simply a case of adding a new repository to your sources.list file. First, add the key to your system:
wget -q -O - http://pkg.hudson-labs.org/debian/hudson-labs.org.key | sudo apt-key add -
Add the following to /etc/apt/sources.list, at the end:
deb http://pkg.hudson-labs.org/debian binary/
Update your package list and install Hudson:
apt-get update apt-get install hudson
Validate that everything installed correctly by making a web request to port 8080 on the server with a web browser. All being well, you should be presented with a short setup screen and Hudson itself.
Configuring Hudson
Now we have a Hudson installation up and running, we can leave the command line for a while and take a quick tour around the Hudson interface. You will see a fairly standard-looking interface with the menu down the left hand side of the screen. Since we have no projects running yet, there is not a great deal to look at. The most interesting section to look at at the moment is ‘Manage Hudson’. Head to the ‘Manage Plugins’ section and click the ‘Available’ tab. We will only require a few plugins to start off with in order to integrate with PHPUnit, so install xUnit and Clover. If you will be using Phing, you also need the Phing plugin. Simply tick the checkboxes next to each plugin to include, and click ‘Install’ at the bottom of the page.
Once the plugins have been installed, Hudson will be restarted automatically and your new plugins will be loaded.
That’s all we need to get started. The rest of the configuration is Project specific, and we can take a look at an example project set up now.
Head back to the dashboard once the installation of the plugins has completed, and click ‘New Job’. Enter ‘Test Project’ as the name of the new Job, and select ‘Build a free-style software project’ as the project type. On clicking ‘OK’ you will be presented with the full configuration screen for your project along with the required settings for each of the plugins you have installed.
Each configuration section comes with it’s own help text, so explore each of the options available to you as you go through the configuration options.
Version Control
Version Control is a large part of the set up of your project in Hudson. Out of the box, both CVS and Subversion are supported as version control systems, however many more can be set up via the plugin system. For this tutorial I am going to use Subversion, so select the Subversion radio button under the Source Code Management section. This will reveal further SVN-specific configuration options which should be fairly self-explanatory.
Enter the location of the project that you want to build under ‘Repository URL’. If your repository requires authentication you will be prompted to supply the credentials that you need. Each time your project builds, it will update the code from this repository prior to running the ‘build’ that we will configure next.
Build Triggers
You have a number of options to configure what triggers a new build. You can read more about each of this by looking at the help options. The most useful is the ‘Poll SCM’ option. At regular intervals, Hudson will check your version control (configured above) to see if any changes have been checked in since the last build. If there are any, the code will be updated and a new build will be triggered.
Select ‘Poll SCM’ and enter ‘*/15 * * * *’ into the Schedule. The notation may be familiar since it is just cron syntax – read the help for more information on what it means if you are not alraedy familiar with cron. This schedule setting will cause Hudson to check your repository every 15 minutes for any changes.
Build
Now for the build itself. A drop-down allows you to configure a variety of build options (extendible by plugins). Whether you are using ant or Phing, select the appropriate build step from the drop down and some extra configuration options will be displayed.
Assuming you are running ant to run your build, you are likely to have a step in your build.xml file that looks like this (dir should be set to the directory within your repository that the tests are run from. Often, this is the location of your phpunit.xml file).
<?xml version="1.0" encoding="UTF-8"?>
<project name="My Project" default="build">
<target name="build" depends="phpunit" />
<target name="phpunit">
<exec executable="phpunit" dir="${basedir}/tests" failonerror="on">
</exec>
</target>
</project>
In order to analyse the results of the build (since PHPUnit will be doing all the work!), you need to instruct PHPUnit to export the build results and code coverage reports in an XML and HTML format that Hudson is able to parse and display once the build step has been completed. This is achieved by instructing ant to create a ‘build’ directory, and adding a few arguments to your ant target. Update your ant build so that the –log-junit, –coverage-clover and –coverage-html commands are being passed in and the build directories are created, like this:
<?xml version="1.0" encoding="UTF-8"?>
<project name="My Project" default="build">
<target name="build" depends="phpunit" />
<target name="init">
<mkdir dir="${basedir}/build/logs" />
</target>
<target name="phpunit" depends="init">
<exec executable="phpunit" dir="${basedir}/tests" failonerror="on">
<arg line=" --log-junit '${basedir}/build/logs/phpunit.xml' --coverage-clover '${basedir}/build/logs/clover.xml' --coverage-html '${basedir}/build/logs/coverage'" />
</exec>
</target>
</project>
You can test that your build runs by just entering ‘ant build’. Assuming everything goes well, check the updated build.xml file in to your source control repository. As we have set up the ant config to run everything we need under the build target just enter ‘build’ in the ‘Targets’ field in Hudson.
If your build.xml is not in the root folder of your repository you can click Advanced and specify the full path to it (relative to the root directory) in the ‘Build File’ configuration options.
Post Build Actions
There are a number of post build actions that are available. For now, we are only interested in two of them (for each of the XML files that our build step has generated). The first one, is ‘Publish testing tools result report’. Select this option and choose ‘PHPUnit-3.4′ from the drop down menu. Enter the path to the built results relative to the workspace root into the ‘Pattern’ configuration option that appears. If you are building from trunk, this will be trunk/build/logs/phpunit.xml (remember, this is the xml file that we passed in to the –log-junit parameter to phpunit).
You should also enable the ‘Publish Clover Coverage Report’ option. Specify trunk/build/logs as the clover report directory, and specify ‘clover.xml’ as the clover file name.
Click save, and manually trigger your build by clicking the ‘Build Now’ option from the menu on the left hand side.
More Plugins for Extending Hudson
PHP Mess Detector
The PHP Mess Detector is a static analysis tool that examines your code and creates a report on various code quality metrics. As with PHPUnit, it is able to generate this report in an XML format that Hudson can parse and display along side your unit test results using the PMD Plugin. Install PHPMD onto your build server from PEAR using the following commands:
pear channel-discover pear.phpmd.org pear channel-discover pear.pdepend.org pear install --alldeps phpmd/PHP_PMD-alpha
You will need to add a target to your ant build file to get it it run phpmd as part of the ‘build’ target. Here is an example of an ant target (paths will need to be adjusted to suite your own project). To add it to the build target, just add phpmd to the ‘depends’ attribute in the ant XML.
<target name="phpmd">
<exec executable="phpmd" dir="${basedir}" failonerror="off">
<arg line="source xml codesize,unusedcode --reportfile '${basedir}/build/logs/pmd.xml'" />
</exec>
</target>
The PMD plugin will add an option to your post build actions in plugin. The only requirement is that you enter the location of the XML file that is produced during the build step trunk/build/logs/pmd.xml.
PHP Code Sniffer and Checkstyle
Most projects have some form of coding standards. You can configure Hudson to report on coding standards violations by parsing the output from running the PHPCodeSniffer (phpcs) with the Checkstyle plugin. If you want to read more about PH Code Sniffer, you can read the manual page or the techPortal tutorial on using phpcs. Install phpcs from PEAR on your build server using the following command:
pear install PHP_CodeSniffer-1.3.0a1
Here is an example of a build target in ant for the phpcs program. Again, add this to your build target if you want to run it and install the Checkstyle plugin in Hudson.
<target name="phpcs">
<exec executable="phpcs" dir="${basedir}/source" output="${basedir}/build/logs/checkstyle.xml" failonerror="off">
<arg line="--report=checkstyle --standard=PEAR --extensions=php '${basedir}/source'" />
</exec>
</target>
PHP Copy Paste Detector and DRY
Code that has been copied and pasted from once file to another if often the cause of quality issues in software applications. The PHP Copy Paste Detector (phpcpd) can help detect it and the DRY plugin for Hudson can display the results for you.
The dependencies for phpcpd require a version of php greater than 5.2.7 so you may need to take steps to upgrade the version of php on your server (at the time of writing, only 5.2.6 is available in debian stable).
pear channel-discover components.ez.no pear install phpunit/phpcpd
<target name="phpcpd">
<exec executable="phpcpd" dir="${basedir}" failonerror="off">
<arg line="--log-pmd '${basedir}/build/logs/phpcpd.xml' source" />
</exec>
</target>
The Continuous Integration Game
As a bit of fun, and as motivation for your team to keep an eye on the Hudson build, you can install the ‘Continuous Integration Game’ plugin. Points will be awarded to users who commit code with tests, reduce the PMD warnings and fix checkstyle issues. Points are lost for introducing problems to the build, and causing the build to fail (the game is quite punishing on this point!).
In Conclusion
I hope this article has provided a good introduction and guide to finding your way around the Hudson CI environment. We’ve covered the technical side of installation here, but not the wider points of continuous integration as a practice – if you would like to read more about the concept from a business perspective, you can download our whitepaper on continuous integration. We have certainly put Hudson to work at Ibuildings and see the benefits of having it every day. Hopefully, this will inspire you to do the same.