Jenkins, CircleCI, Travis, GitLab-CI: Getting Started with Continuous Testing & Integration

by Tom Helvick

Continuous testing and integration are increasingly becoming must-haves in modern software development. Significantly, continuous testing ensures that all code has been thoroughly tested before it can reach staging or production. As a result, companies can deploy with greater confidence and developers get a faster feedback loop on potential flaws in the code they check in.

For this reason, most enterprises are looking to implement continuous testing and integration. However, many companies don’t know where to start. Continuous integration (CI) can be complex and confusing, especially for existing applications with large code bases and scattered test coverage. Nevertheless, getting started with CI doesn’t have to be difficult. In fact, most companies can set up barebones CI in a matter of minutes. While it won’t be perfect at first, getting started and iterating from there is the key to eventually reaping the benefits of automation.

This article will cover the benefits of continuous testing and integration, the popular options for implementing CI, and a quick primer on how to take the first steps.

Why Test Continuously?

Before we dive into the nuts and bolts of implementing CI, it’s important to consider why you want CI in the first place. Indeed, different companies have different priorities for their CI pipelines. Correspondingly, you’ll want to consider your goals before you read the rest of this article. Your goals will help you choose among the various options.

In general, the benefits to CI are speed, focus, and confidence.

1. Faster feedback in the development cycle

Because tests run immediately every time a developer checks in code, the developer knows immediately if their changes broke the build. Formerly, developers might have written code in isolation and then tried to combine everyone’s changes at the end. Before release in the final hours, all hands had to be on deck to fix bugs and identify conflicting changes. With CI, your main build should always pass all tests. This is because CI guarantees that breaking changes don’t make it to the master branch (if you’ve written comprehensive tests).

2. Reduced multi-tasking

Related to faster feedback is the concept of reduced multi-tasking. When the feedback cycle is long, developers move on to new stuff while they’re waiting for the tests to run. Consequently, developers often forget how they implemented something last week or even yesterday. As a matter of fact, Microsoft found that bugs that went unaddressed for a while took much longer to resolve. In contrast, when developers were notified of bugs immediately, the time it took to squash those bugs was greatly reduced.

3. Decreased business risk on software release

Since CI guarantees all code has been tested, you can be more confident in release decisions. Of course, you’ll need to have confidence in the coverage and depth of your test suite. However, the decision to deploy becomes much simpler. Indeed, CI is the first step toward continuous delivery and deployment, the foundation of DevOps and a key to agile workflows.

Hosted vs On-Premise

CI solutions come in two flavors: hosted or on-premise. Both have pros and cons.

Hosted solutions are cloud-based. In general, they track an online repository and notice any changes to that repository via webhooks. When a developer commits a change to that online repository, it automatically initiates a build on the CI solution. Hosted solutions are generally quick to set up, easy to navigate, and cost-effective. However, they include fewer features and nitty-gritty customizability than on-premise solutions.

On-premise CI involves spinning up your own test server locally. Doing so gives you more fine-grained control over how the build is configured and run. You can also add endless plugins to your CI tool for even more features. However, these plugins can quickly become unruly and difficult to maintain. In addition, installation can be a challenge and their UIs are generally less intuitive than their hosted competitors.

Both options allow you to track the status of a branch on a remote repository. From there, you can choose to only merge branches that are passing CI tests. Furthermore, you can set up your master branch to automatically deploy a new version when all tests pass after merging changes.

Popular Options

There are a handful of commonly recommended CI providers. No matter what it is, chances are one of these will work for your project:

1. Jenkins — On-premise. Rich plugins, largest community, Java-based so runs on any platform. Config and UI are popular complaints
2. GitLab — On-premise. Easier setup and maintenance than Jenkins, not as rich a plugin library but supports most common needs, scalable on parallel machines, best Docker support
3. CircleCI & Travis — Hosted. (These two options are very similar, so I’ve grouped them.) GitHub integration via webhooks, automatic deployment, nice online interface, config matrix via a yml file

How to Get Started

To get started with CI, you’ll first need an application and some tests for that application. If your test suite is complicated, then try writing a simpler set of tests for your first shot at CI. Eventually, you can add the rest of your tests. However, reducing complexity should be your goal for your first CI run.

If you’ve chosen a hosted solution, then setting up CI is as simple as linking your repository and providing a config file (usually .yml) with instructions on how to run your application and its tests. On the other hand, setting up an on-premise solution requires more up-front work. In short, you’ll need to install the CI application and any plugins and then follow the config instructions from your CI provider.

Once you’ve set up your application to trigger a test build with any new commits, you can focus on writing good tests. Obviously, your CI suite is only as good as the tests you write for it. Undeniably, CI is a valuable process for building software, but it isn’t a magic bullet. Therefore, use CI as a way to strengthen your organization’s overall testing culture and move toward embracing agile development best practices.

About Intertech

Founded in 1991, Intertech delivers technology training and software development consulting to Fortune 500, Government and Leading Technology institutions. Learn more about us. Whether you are a developer interested in working for a company that invests in its employees or a company looking to partner with a team of technology leaders who provide solutions, mentor staff and add true business value, we’d like to meet you.

…

Originally published at Intertech Blog.