There are numerous guides available on line for setting up a fresh install of Jenkins with numerous different configurations but i wanted to create a chef recipe that would give me a clean Jenkins install with my required project dependencies (excluding rvm) at the click of a button.

The reasons the excluding RVM in the chef process, even though the chef-rvm cookback is fantastic having used it before in the creation of a Rails stack is that i just could get system level RVM to play nice with Jenkins and user specific RVM (to the Jenkins user) to automatically install without asking for the Jenkins password. Better minds will probably be able to solve this issue but i opted for a different approach.

The CookBook

My build cookbook and recipes can be found over on GitHub at [URL]. It contains everything we require (except rvm) for building our rails apps, Postgres for the postgres C headers for the pg gem, a javascript runtime in the form of node.js for rails 3.1 and greater asset compilation, a web server for accessing Jenkins, libqt4 and Xvfb for headless running of our integration specs with capybara webkit.

Clone or download the chef repositry from [URL]
add in your own .chef/knife.rb and .pem files or make sure they can connect to your chef server
run berks install and berks upload to collect together all the associated cookbooks then push them up to the hosted chef repo
run rake install to upload our custom cookbooks and roles
then create a new instance of EC2 that will automatically be built with our jenkins build recipes by running.

sudo

this command requires a few config options to be added to your knife file to achieve the brevity. I wrote about this in a previous post here

This final command will set about creating a new EC2 micro instance with Ubuntu 12.04 AMI in Europe and then install all the necessary packages for running our standard rails stack. It will also setup a reverse proxy for nginx so that jenkins will be available on on completeion at the EC2 instances Public DNS address.

If all goes well the instal takes around 20 minutes and at the end you should see a print out as such.

ecX-XX-XXX-XXX-X.eu-west-1.compute.amazonaws.com Chef Client finished, 91 resources updated
ecX-XX-XXX-XXX-X.eu-west-1.compute.amazonaws.com

Instance ID: i-XXXXXXXX
Flavor: t1.micro
Image: ami-7962730d
Region: eu-west-1
Availability Zone: eu-west-1a
Security Groups: default
Security Group Ids: default
Tags: {"Name"=>"i-XXXXXXXX"}
SSH Key: chef-test
Root Device Type: ebs
Root Volume ID: vol-XXXXXXXX
Root Device Name: /dev/sda1
Root Device Delete on Terminate: true
Public DNS Name: ecX-XX-XXX-XXX-X.eu-west-1.compute.amazonaws.com
Public IP Address: XX.XXX.XXX.X
Private DNS Name: ip-XX-XX-XXX-XXX.eu-west-1.compute.internal
Private IP Address: XX.XX.XXX.XXX
Environment: _default
Run List: role[jenkinsbuilder]

Copy your public DNS address ecX-XX-XXX-XXX-X.eu-west-1.compute.amazonaws.com paste into a browser, and you should be greated by jenkins loving face.

Jenkins Welcome

A few caveats

We need to change the HTTP_HOST value in /etc/default/jenkins to be 127.0.0.1 as it is set by default to the DNS Name which means the deamon doesnt work to reload the app with nginx front end.
Secondly we need to jenkins ALL=(ALL) NOPASSWD:ALL to /etc/sudoers so that Jenkins isn’t asked for a password (jenkins password required for ‘apt-get —quiet —yes update’:)
Postgres extensions arent installed and any builds which use them will fail. So if your using an extension like hstore it needs to be activated and this gist gives a good run down of the commands to run.

Building our Rails Project

Setting up RVM

The instructions for installing rvm with can all be found with more detail at here, so the below is mearly a quick overview of the commands i ran to get rvm set up.

    sudo su -s /bin/bash jenkins #Loggin in with full terminal as jenkins 

    cd ~ #change to jenkins home dir

    \curl -L https://get.rvm.io | bash #install RVM the opening slash is important and delibrate

    # Add the following two lines to ~/.bashrc 

    [[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"
    PATH=$PATH:$HOME/.rvm/bin # Add RVM to PATH for scripting

    # exit the shell and log back in 

    type rvm | head -1 # should show rvm is a function

    # add the following flags to ~/.rvmrc

    rvm_install_on_use_flag=1
    rvm_project_rvmrc=1
    rvm_gemset_create_on_use_flag=1

Again more explanation can be found over at the rvm instruction pages. here

Pulling from Heroku

To pull from heroku ie use heroku as the SCM we need to add the generated SSH key to a user associated with the project.

sudo su -s /bin/bash jenkins

cd ~

cat ~/.ssh/id_rsa.pub

The output from the cat command needs to be copied and pasted into the ssh-keys for a user on heroku.com or via heroku cli heroku keys:add ~/.ssh/id_rsa.pub

The next thing is to make sure heroku is added to jenkins ~/.ssh/known_hosts, the easiest way to do this is just git clone one of the projects for user the key was just added to and type yes when asked about the host.

Setting up Jenkins

Now that Jenkins will boot up and we are ready to pull from our SCM provider its time to setup security, install some plugins and set some global config variables so make life easier.

Security

Jenkins security is pretty simple and the docs do a great job of demonstrating how to set up a standard username/password login system with you as the super user. There is no need for me to duplicate this so head over to Jenkins Wiki and follow their instructions.

Plugins

The following plugins are my standard goto’s for a new jenkins install

git – Adds git support to Jenkins SCM section. A global user name and email address will need to be configured in the global settings panel.
thinbackup – Regularly backup global config and project configs
chucknorris – Chuck Norris banter on failed and successful builds
rubyMetrics – Allows the interpretation of RCov coverage reports
heroku – Interact with heroku
email extension – Extends Jenkins built in e-mail.

Setting up the Rails Project

Getting good reports

When running rails tests within Jenkins the output isn’t formatted in such a way that Jenkins really knowns what to do with it, so a failed test run and a successful test run are seen as fundamentally the same thing. To alter this behaviour need to add the ci_reporter gem to to your projects gem file. The same applies to the standard code coverage gem simplecov, the output produced can be seen in the Jenkins workspace, but you dont get the nice tracking graphs. An easy way to fix this is with the simplecov-rcov gem which allows simplecovs output to be formatted like rocvs which can be picked up by the rubymetrics plugin in Jenkins.

All you need to do is add the following lines above SimpleCov.start

require 'simplecov'
require 'simplecov-rcov'
SimpleCov.formatter = SimpleCov::Formatter::RcovFormatter
SimpleCov.start 'rails'

The best way to get the correct formatted output is to set up rake task using RSpec::Core::RakeTask in ‘lib/tasks/spec.rake’ that allows you to run all specs with the command rake all

require 'rspec/core/rake_task'
require 'ci/reporter/rake/rspec'

RSpec::Core::RakeTask.new(:all => ["ci:setup:rspec"]) do |t|
  t.pattern = '**/*_spec.rb'
end

Separate Environment

The last thing i did was setup a new environment for the Jenkins user based of the default test environment. First I create a config/environments/jenkins_test.rb, add this to any bundler groups that test env was part of in config/application.rb and then leave out setting up a entry in database.yml as this will be done during the Jenkins build process

Project Setup

Advanced Project Options

Set up quiet time option in the build when using heroku deploy hooks as they can fire before the last commit is available

Build Triggers

Trigger builds remotely through heroku deploy hooks.

Build

The current stock build script for a rails project

#!/bin/bash 

# Jenkins is installed as a service account when its running builds so it isn't 
# using a login shell. So since RVM was installed to the user and is setup through .bashrc 
# this doesn't happen so we need to load RVM right after the shebang   

source "$HOME/.rvm/scripts/rvm"

# Set the global rails env
export RAILS_ENV=jenkins

# Load RVM with a ruby and a gemset, if neither are present they will be installed
rvm use 2.0.0@brandtone

set -e

bundle install

# Duplicate a stock database yml file and the change our database name
cp ~/database.yml config/database.yml
sed -i 's/database_name/brandtone/g' config/database.yml

# Create if not present and migrate out DB
rake db:create RAILS_ENV=jenkins_test
rake db:migrate RAILS_ENV=jenkins_test

# run all tests in a virtual frame buffer thats auto numbered to prevent clashes
xvfb-run --auto-servernum bundle exec rake all

Post Build Actions

Standard Post Build Actions

Activate Chuck Norris
Publish JUnit test result report
Publish RCov Report
Send Email Notifications

Jordan Rogers-Smith

Coding away day to day.

Setting Up Jenkins With Chef for Rails Project