Jenkins in a Docker container

After having our Sonatype Nexus repository running in a Docker container (see my previous post), the next part of our infrastructure I wanted to move into a Docker container was our Jenkins build server.

The most difficult part of this, was to get the Android SDK installed inside the Docker image. After downloading the compressed SDK you have to run the android installer from the tools folder. This installer downloads the requested Android versions and asks for accepting the different license agreements – which is a problem because the building the docker images is non interactive. After a lot of trial and error, I found the Android SDK Installer project on github which provides the necessary scripts.

Here is my resulting Dockerfile using the Android SDK Installer:

The steps in this Dockerfile are

  • installing required libs for the Android SDK via apt
  • downloading the latest Jenkins WAR file
  • using the Android SDK Installer to install the required parts for building our Android apps
  • defining the ANDROID_HOME environment variable
  • defining an entrypoint using the war file for starting Jenkins
  • exposing port 8080 for accessing Jenkins
  • defining a volume for persisting the Jenkins work directory

With the image generated by this Dockerfile it is now possible to start up a Jenkins build wherever I want. To make the running Jenkins docker container available via Apache on Port 80 I use the following reverse proxy config:

The jenkinsPort is defined in an apache config file as 8080.

I am using a data volume for persisting the /jenkins folder as explained in my last post under “Data volumes”.

Because Jenkins allows to install everything else that is required afterwards via the web UI (JDKs, Maven, Gradle, Plugins, …) this basic image is enough. Another great improvement in Jenkins is the ability to configure credentials, like SSH keys and username/password combinations, via the web UI. This way it is not required to stored the secret stuff inside the Docker image anymore.

Again, like for the nexus repository in my last post, it is very nice to update Jenkins to the newest version by just executing docker stop, docker build and docker run.

For our Xcode builds we use the Jenkins config that is executed via SSH access to a Mac mini server. Nothing Jenkins specific had to be installed on the Mac mini, Java, Xcode and a SSH access was enough.

Docker infrastructure

Since I saw the 5 minute long video “The future of Linux containers” last year in April, in which Solomon Hykes presented Docker the first time at PyCon, I wanted to use Docker for running my server side software.

The first thing I thought about, was using it for running our time tracking product timr. It is currently deployed on multiple hosts with multiple tomcat instances and a MySQL database. Docker would make it easy to spin up more instances and to deploy new versions.

We are also hosting server applications for our clients. Sometimes only for testing but in some cases also for production. Most of them are JVM based and it is a tedious task to set up yet another tomcat instance and apache virtual host for every new project. Docker would help here too.

Another topic docker could solve for us, was creating test environments triggered by jenkins builds. Currently we have jenkins jobs that can be run by the QA team to deploy a new version on the test server, but sometimes different testers want to have different versions deployed. I would be nice to create a new test instance from a jenkins build, accessible via an auto generated URL.

Breaking it down

After thinking too long about how to tackle all those tasks at once I thought I should start small. So I started with creating a Docker container for running our Sonatype Nexus artefact repository. Here is the resulting Dockerfile:

As you can see it is based on a docker image called troii/java7 which I will show you later. To install nexus inside the image it does the following:

  • downloads the latest sonatype nexus version into /usr/local
  • extracts the TAR file and deletes it
  • create a symbolic link /usr/local/nexus to the newly created folder
  • uses sed to adapt the nexus script to run as user root
  • changes the context path from /nexus to / in the nexus.properties file

The image exposes the port 8081 (the default port nexus uses) and adds a volume /usr/local/sonatype-work to persist the working directory of the repository. With this Dockerfile it is now possible to start up my nexus server wherever I want within seconds.

Currently it is running on an Ubuntu 14.04 root server at Hetzner. I made it available via apache reverse proxy with this config in a virtual host:

The nexusPort (in this case 8080) is defined in another apache config file where I configure all the ports for my reverse proxies.

Data volumes

If I would only run the image created by the Dockerfile before, keeping the persistent data between stopping and starting containers would not be very intuitive. Though I defined a volume for the directory sonatype-work (which makes it persistent) I would need to start the next container with the option –volumes-from to reuse the volume from the last nexus container I stopped.

Luckily there is an article called “Managing Data in Containers” that handles exactly this topic. What I did was creating a so called “data only” container that is used in the –volumes-from option every time to start the nexus container.

This leads to two images/containers being involved in my nexus docker setup, which raises the question on how to manage the startup configs for my docker containers. Those command lines with all the correct options for starting the container can get very long – should I create separate scripts just to start the containers?

Base images

As I wrote before the nexus docker image is based on the troii/java7 docker image which looks like this

It installs the Oracle JDK 7 and is based on the troii/base image which looks like this

The base image uses Ubuntu 14.04 Trusty and does the following on top of it

  • it configures a nearby ubuntu mirror
  • sets some environment variables, timezone and locale
  • installs zsh, vim, git and other stuff I like to have on the command line

First I started with one big Dockerfile but over the time I started to split them up in this way. The base image is used for every docker image I created in the last weeks.

By creating a separate image with Java installed, it is easy to update the Java version for containers base on it. Just by simply building the troii/java7 image again it downloads the latest version. When a container base on it is rebuilt the Java version is automatically updated too.

Conclusion

So far one of the biggest benefits of “dockerizing” our infrastructure was that the setup and configuration is reproducible stored in git repositories. Creating the Dockerfile really documents how the services are set up.

Another thing I noticed that is really nice about having nexus run in a docker container like this is that I just have to run three simple steps (docker stop, docker build, docker run) and I have updated nexus to the newest available version.

What is next

I want to start a series of blog posts with this one about my ongoing effort on moving more and more to a infrastructure based on Docker. There are still many questions left but some have already been answered … stay tuned!

Why Podcasts are awesome

Am Mittwoch den 16. Juli 2014 fand wieder ein Technologieplauscherl statt, dieses Mal waren wir bei Netural eingeladen. Ich durfte dort einen von vier Vorträgen halten, der Titel meines Vortrages: Why Podcasts are awesome

10446110_763889400298907_7621158855081489775_o

Foto by Klaus Fischer

2014-01-08_1389217354Wie der geneigte Leser meines Blogs (oder meiner anderen social Media Kanäle) ja schon mitbekommen haben dürfte, produziere ich seit ca. einem Jahr gemeinsam mit meinem Freund und Kollegen Andrè Steingreß das Donau Tech Radio.

Der Untertitel unseres Podcasts lautet “österreichischer Podcast rund um Software Entwicklung”. Wir reden über Themen die uns während unserer Arbeit in der Software Entwicklung unterkommen aber auch über News und Gadgets.

Den Vortrag beim Technologieplauscherl habe ich gehalten, weil ich einerseits mehr Leute von Podcasts begeistern will (mehr Hörer unseres Podcasts) und andererseits gerne Leute dazu bringen will selbst Podcasts zu produzieren (mehr Podcasts aus Österreich).

Insgesamt war das Plauscherl bei Netural wieder super – gute Location, super Verpflegung und lauter “Gleichgesinnte” anwesend ;-)

Die Slides zu meinem Vortrag hab ich auf Speakerdeck gestellt und hier embedded:

Apple introduces Swift and does Bret Victor style demo of Playgrounds

When I watched the video of Bret Victor’s presentation “Inventing on Principle” I was really fascinated about how software development tools could look like – take the time, you will enjoy this presentation:

Bret Victor – Inventing on Principle from CUSEC on Vimeo.

Today Apple presented Swift and the new developer tools they are providing around it. They demonstrated a feature called Playgrounds that looks a lot like Bret showed:

This is really exciting, I am looking forward to experiment with this.

Solar Roadways and Google’s self driving car

This week two videos went through my news feed that fit together perfectly:

1. The solar road

Wow, this looks really like it could solve a lot of our current problems. I am sure there are still many thing that need to be solved but we can support the project at Indiegogo.

2. Google’s self driving car

I can see those two combined in future where we get the electrical power we need for our (self driving) cars directly from the road we are driving/standing on.

Outlook 2013 does not trim Internet calendar URL

For our time tracking software we provide a feature that allows users to show their time recordings in their calendar application. Therefor we provide a special URL that includes a unique token to allow access to the entries of the user. The URL looks like

https://troii.timr.com/timr/cal/workingtime-1b9119c3-42a2-a5d4-5e97b8f4cda2.ics

It includes a token that authenticates the user and does not need extra login information to access the calendar.

Lately we had some support requests saying that Outlook 2013 asks for a username and a password when using this URL to subscribe to a “internet calendar”. We looked into our log files and found the following:

112.xx.xx.xxx - - [11/May/2014:06:30:00 +0200] "GET /timr/cal/workingtime-1b9119c3-42a2-a5d4-5e97b8f4cda2.ics%20 HTTP/1.1" 200 15217 "-" "Microsoft Office/15.0 (Windows NT 6.2; Microsoft Outlook 15.0.4605; Pro)" 874863 "-" "-" 0

The interesting thing here is the %20 at the end of the URL – it seems there is a space at the end of the URL. We thought it is not possible to add a space to the end of an URL because if you do this in the URL field of a browser it always strips the space character from the URL.

As it turns out this is not the case with Outlook – when you enter an URL that ends with space character it is sent to server like that, which is invalid.

I hope this is a bug and Microsoft fixes it in the next versions because I can not see why this should be a feature.

Fast am Ziel

Nachdem ich letzte Woche auf die neuen powerSpeed Tarife der Energie AG für Privatkunden hingewiesen wurde (hier zu finden), habe ich wieder Mal bei der Energie AG angerufen und um einem solchen Anschluss gebeten.

Der freundliche Herr am Telefon hat mir versprochen sich meine Situation anzusehen und sich wieder zu melden. Diese Woche läutete dann das Telefon und der selbe Herr war dran. Ich habe mich schon auf die nächste Absage eingestellt, doch zu meiner Überraschung und großen Freude, hat er mir mitgeteilt dass ich einen Glasfaseranschluss von der Energie AG bekomme.

Er hat mir dann auch erklärt wie das ablaufen wird. Das hat gezeigt, dass er sich die örtlichen Gegebenheiten schon genau angeschaut hat. Die notwendigen Grabungsarbeiten sollen im März stattfinden. Ich habe dann auch noch sofort einen Vertrag per email erhalten, unterschrieben, gescannt und zurückgesendet.

Das ganze ist für mich ein schönes Weihnachtsgeschenk. Endlich bekomme ich genau den Internetzugang für den ich schon lange gekämpft habe  (fiber to the home). Hoffentlich kommt da nichts mehr dazwischen *fingers crossed*

In diesem Sinne frohe Weihnachten!

Österreich unter den Verlierern beim Ausbau des Glasfasernetzes

Wer mich und meinen Blog kennt, der weiß dass, ich mich hier schon das eine oder das andere mal über den miserablen Zustand der österreichischen Internetzugänge beschwert habe. Heute bin ich auf einen Spiegel Artikel von Sascha Lobo mit dem Titel Ein digitalpolitisches Armutszeugnis gestossen. Darin geht es um verschieden Versäumnisse der deutschen Bundesregierung im Bereich Digitalisierung und es kommt under anderem folgender Punkt vor:

die dramatisch vernachlässigte Infrastruktur. Während andere Länder zweistellige Milliardenförderungen in Glasfasernetze pumpen, ignoriert die Bundesregierung, dass die Unternehmen allein niemals die ungeheuren Investitionen stemmen könnten – und wenn, dann nur mit schädlichen Hilfsmitteln wie der Aufgabe der Netzneutralität oder fatalen Drosselungsdummheiten. Das Resultat: Die Auflistung der 22 europäischen Volkswirtschaften mit den meisten Glasfaseranschlüssen beinhaltet Deutschland gar nicht erst. Weil die Darstellungsgrenze bei einem Prozent liegt. Das klitzekleine Russland kommt auf 14 Prozent Haushalte mit Glasfaser, das superreiche Bulgarien auf 17 Prozent, und in Litauen verfügen mehr als doppelt so viele Haushalte über Glasfaser wie in Deutschland. Und zwar in absoluten Zahlen.

Wenn man sich die Auflistung in dem Bericht “Winners and losers emerge in Europe’s race to a fibre future” ansieht sucht man Österreich unter den Top 22 beim Glasfaserausbau Europas natürlich auch vergeblich. 

Auch der Aspekt der morgen anstehenden Nationalratswahl macht mich nicht zuversichtlich dass sich hier mit der nächsten Regierung viel tun wird.

Named access on the Window object in Safari

downloadThis week we noticed that the user administration of our time tracking app timr behaved different on Safari.

I debugged into that issue and found out that a condition in JavaScript behaved different than in other browsers:

if (assignedGroupTable) ...

The variable assignedGroupTable was defined some lines before with

var assignedGroupTable;

and the condition should be false on the first call of that function, but in Safari it was true. The debugger showed that the value of assignedGroupTable seemed to be a DIV – there was a DIV in the page with an ID assignedGroupTable, but it had never been assigned to that variable. It seemed like Safari automatically assigned the DIV, with the ID, that was equal to the variable name, to the variable.

After a quick search, I came to this article about DOM Element References as Global Variables and I was really surprised, about what I read there. It seems like all modern browser implement the Named access on the Window object, which allows you to access elements via the ID or NAME attribute, simply by using it like a variable in JavaScript.

So why did our code behave different in Safari? I adapted the example the article uses to find the difference in the behaviour of Safari.

On modern browsers (Chrome 29, Safari 6, Firefox 22, IE 10) this logs the button element to the console. But when making a small change the browsers behave different:

In Chrome, Firefox and IE this logs undefined to the console, only Safari still logs the button element!

There were two ways to fix our problem:

  1. rename the ID of the DIV or the variable so that they differ
  2. define the variable in the JavaScript with
    var assignedGroupTable = null;

    setting the value explicitly to null

Wo decided to do the second because it required fewer changes.