User Tools

Site Tools


Sidebar

Navigation

Join us by

user mailing list
devel mailing list


More news...

RSS

ci:tutorial

Installing Jenkins

Jenkins is easy to install, it can be run either as a stand-alone application, or deployed as a conventional Java application. Before, the installation, please verify that a JDK better than the version 5.0 is installed in the machine.

Run as Stand-alone Application

The first option is a simple way to get start. Download the latest and greatest WAR archieve from the official website, and place it in an appropriate directory as /usr/local/jenkins. Then, open a console in the directory containing the jenkins.war file and run the following command:

$java -jar jenkins.war
webroot: $user.home/.jenkins
Jan 04, 2013 2:34:35 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Jenkins home directory: /home/ibm/.jenkins found at: $user.home/.jenkins
Jan 04, 2013 2:34:38 PM winstone.Logger logInternal
INFO: HTTP Listener started: port=8080
...
Jan 04, 2013 2:34:50 PM hudson.WebAppMain$2 run
INFO: Jenkins is fully up and running

Jenkins should now be running on port 8080 at localhost.

However, running the stand-alone application is not recommended since Jenkins is launched manually. As a CI server, Jenkins should start automatically when the machine is recovered from any accidental shutdown.

Installed as Conventional Application

The installation as conventional application is recommended since it allows Jenkins to be launched when the phisical machine starts. In Linux, root access is necessary for the installation. Native binary package is available for Fedora system. The first step is to set up the repository as follows:

$ wget -O /etc/yum.repos.d/jenkins.repo http://jenkins-ci.org/redhat/jenkins.repo
--2012-12-21 10:37:35--  http://jenkins-ci.org/redhat/jenkins.repo 
Resolving jenkins-ci.org (jenkins-ci.org)... 63.246.20.93
Connecting to jenkins-ci.org (jenkins-ci.org)|63.246.20.93|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://pkg.jenkins-ci.org/redhat/jenkins.repo [following]
...
2012-12-21 10:37:36 (15.3 MB/s) - `/etc/yum.repos.d/jenkins.repo' saved
[75/75]
$ rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key

Next step is to install the package:

$ yum install jenkins
Loaded plugins: langpacks, presto, refresh-packagekit, security
google-chrome |  951 B     00:00
jenkins |  951 B     00:00
...
Verifying  : jenkins-1.494-1.1.noarch 1/1
Installed:
  jenkins.noarch 0:1.494-1.1
Complete!

This will install the latest version of Jenkins into the /usr/lib/jenkins directory. The default Jenkins home directory will be in /var/lib/jenkins.

Now the Jenkins can be launched by the service command:

$ service jenkins start 
* Starting Jenkins Continuous Integration Server jenkins                [ OK ]

Then, Jenkins will start automatically every time the machine reboots. It can be closed by the service command:

$ service jenkins stop
* Stopping Jenkins Continuous Integration Server jenkins                [ OK ]

Please note that the installation will create a 'jenkins' user in the system. This 'jenkins' user cannot access the file in the software developer's home folder. On the other hand, the software developer who does not have root rights cannot modify the content in jenkins' home directory.

Installing BlobSeer

The detailed instruction of the installation is described in Tutorial. The only difference between the installation on a personal computer and the CI server is the location of libraries (boost, Berkley DB and libconfig,). Since Jenkins does not have rights to read the files in software developer's home folder, the libraries should be installed in the location where all users in the system can access as /local repository.

Setup Jenkins

The generic instruction of Jenkins configuration is available here. To build and test BlobSeer, some specific plugins and configurations are necessary.

Global Configuration

Two special plugins are required to build and test BlobSeer: CMAKE and SSH. Installing the plugins are straightforward. Choose the Manage Jenkins option from the left top panel, then enter the Manage Plugins screen. Select Hudson CMake Plugin and Jenkins SSH plugin, then finish the installation. After the installation, CMake option is already available in the Build section of job configuration.

To configure SSH connections, first enter the Configure System screen, then fill the SSH remote hosts section. For BlobSeer, two SSH connections are required, one connects to the CI server itself to execute local deployment, the other connects to Grid5000 to test BlobSeer on a cloud environment.

To set up SSH connecting to local CI server, software developer should enter his own user name and password that are used to access the local machine.

To set up SSH connecting to Grid5000, …

Job Configuration

Since BlobSeer envisions large scale data storage, both the local test and the test on grid are important. To this end, two jobs, local build and grid build, are designed so that they do not interference with each other.

Local Build

Local build aims at quick and simple check of recent modifications on the software. Therefore, Build Triggers is set to both nightly and post-commit build.

The total build and test includes three parts: CMake build, execute shell and execute shell script on remote host using ssh.

The CMake build can be set as follows:

As it is mentioned before, developer who does not have root rights cannot access the released executions in Jenkins' home folder. The following script is used to copy the built BlobSeer into the /local repository so that the developer can deploy and test BlobSeer without difficulties.

The last part deploys BlobSeer on CI server and test all the functions. To deploy BlobSeer, it is necessary to have the SSH connection to localhost using rsa key pairs. However, it is not possible to create the pair for jenkins user because the Jenkins platform is not an interactive terminal. The solution is to access to the developer's account and let the developer's account launch the deployment. Of course, the developer should be able to connect to localhost without typing a password. Therefore, the last step is set as follows:

The script test-all.sh deploys BlobSeer. Then, it checks if the four components (version manager, provider manager, data storage provider, meta-data storage provider) of BlobSeer is well launched by invesgating all process running on the machine. Once the script finds that all components are running, it triggers the basic tests including unaligned read BLOB, create, write, read, append, clone and file upload. Unsuccessful deployment or any failed test will immediately lead to an non-zero SSH exit-status. This will yield a failed build process in the end. An email that details the problem will be sent automatically to all concerned developers if the build is marked as FAILED.

Grid Test

To check the health of BlobSeer on Grid5000, it is necessary to build it on the grid. Therefore, the local CMake building is evicted. The first step of build is still to copy the BlobSeer source file to a directory where software developer can access.

…figure

Then, a series of actions are executed on the grid:

  1. Copy BlobSeer resource to the grid (the destination directory on grid is $HOME/blobseer-ci-build-daily-grid)
  2. Build BlobSeer on the grid
  3. Reserve the resource on grid (it is done by auto-reserve.sh)
  4. Wait till the resource are available and execute the test (it is done by wait-and-deploy.sh)

…figure

The script auto-reserve.sh works as follows. It first check the job name BlobSeer-CI. If the job already exists, the script terminates. Then, the process goes directly to the deployment. Otherwise, it searches the available resource on the grid, and makes the reservations as soon as there is enough resource in the grid. Thereafter, it writes into the temporary file job-sub-rec.txt the detailed information of the reserved job.

The script wait-and-deploy.sh retrieves the job information from job-sub-rec.txt. It extracts the start time of the job, and repeats to check the current time on the grid. Once the job starts, it launches the deployment and tests of BlobSeer. How to deploy BlobSeer on Grid5000 is given by this tutorial

Jobs Configuration

Local job configuration in jenkins are now on SVN here

The job just checkout this script and run it.

For example:

cp scripts-ci/Hadoop-BlobSeer-File-System-Monthly-Grid-for-General-Application-Hadoop121.sh .
chmod +x Hadoop-BlobSeer-File-System-Monthly-Grid-for-General-Application-Hadoop121.sh
./Hadoop-BlobSeer-File-System-Monthly-Grid-for-General-Application-Hadoop121.sh
ci/tutorial.txt · Last modified: 2015/01/20 17:01 by lcloatre