How to Dominate Continuous Integration with These 18 Tools
Continuous integration is great and I love it. You can even practice it without any additional tools, but you will quickly realize that it is not very effective.
If you are not using a version control system, you will find that it is a pain for more than one person to work on your code base.
You will also find it difficult to revert to old code when your new work has issues. And these are just a few of the problems that you will encounter.
What about testing? You might run your tests manually, but it is so easy to forget to run them, especially after you’ve made just a tiny change (and then this tiny change brings your whole web service down for hours).
Even if you always run your tests and never forget them, you might still need to find out when was the last time your tests didn’t pass, which will help you pinpoint when a problem first occurred.
Then there is the question between tools hosted on your premises or those hosted in the cloud? Would you rather have more control or easier maintenance?
But it doesn’t stop here. You might select the best place to store your code, then the best place to run your test and finally the best way to deploy your code to production or the best way to build your app.
Even then you have to make it all work together and some tools are easier to work with others.
This entire process is what I call the continuous integration pipeline. Let’s look at what a continuous integration process might look like
- Make a small change to your code
- Run tests locally
- Commit your code to your local branch
- Make a pull request (PR)
- Automatically run tests on your commit (often more tests then those run locally)
- Code review by another team mate
- Merge your code
- Automatically run tests to ensure that everything is still working
- You change is accepted. Repeat.
It is not complicated, but it is not a single step, and as human being when in a hurry or under stress (the site is down we have to push the fix ASAP), it is easy to miss one of them and then create more problems out if it.
It would be great, if you can setup the tools to automate and facilitate your process as much as possible so that you are free to focus on what is important for you.
Version Control System
Why do you need a VCS? Version control systems help you in a lot of ways.
First, they help you by actually keeping multiple versions of your code, just as the name suggests. You can always look at how something worked before and even revert to it if it was a better solution.
Second, you can also make experiments like trying an alternative solution to a problem, while still keeping the current code available.
Maybe one of the biggest advantages of using a version control system is that it makes working with multiple developers on the same code base, much easier.
To the extent that, Google has just a single very large repository and tens of thousands of developers are working from it simultaneously on all Google products. Other companies are following that approach, too.
Git
At the time of writing, the most popular version control system is git. It is the de facto standard for any new startup and for many enterprise companies, too.
It has been created by Linus Torvalds, the creator of Linux and it is extremely fast. In addition to Linux, it works perfectly on Mac and Windows, too.
It is very easy to setup and use, and especially easy to make multiple branches, each for different kind of work. It is what you call a decentralized system. Every person can clone or get changes from the repository of every other without the need of a central server.
In practice, most teams usually prefer to have a central place to store their code, and this makes continuous integration easier.
Mercurial
Mercurial is another popular alternative, very similar to git, having also a very similar back story. It works in pretty much the same way, but is built in Python, unlike Git which build in C.
SVN
SVN is the old kid on the block and some projects still use it.
I personally prefer Git above any other for its ease of use, wide adoption by projects and companies, and speed. Pretty much any where you are going to work it is very probably that you will use Git.
Hosted and Self-Hosted Version Control System
It is true that you can just install the git binaries on every developer machine, and maybe on a central machine and be done with it. Then every developer can sync with the other machines.
This will work, but you will still have a lot of pain when it comes time for code review. You will have to tell your reviewer to look into your code and then manually send him the changes. Which he then needs to apply and compare, by hand.
This will be done over chat or email and thus losing the trail of it.
Instead there are multiple solutions, hosted and self-hosted, which build upon Git and make working with it even easier.
Gerrit
Gerrit is an open source, on premises, self-hosted, solution for Git that was initially developed by Google. It was very similar to an internal tool that was used by Google at the time but could not be open sourced.
It is very good. In our team we’ve used it for years. It makes code reviewing a breeze and you can relatively easily automate different work flows with it.
However, as any on premise solution, you are the one responsible with doing backups and upgrades. Integration with other tools is not hard but requires much more work. Also, it does not share users with other systems, so a user must have an account on Gerrit and then a different account on another system he uses.
Gerrit is still very popular among many projects. Maybe its biggest user is the Android OS.
GitHub
Arguably the most popular Git based solution. Pretty much every open source project is hosted on GitHub, or at least has a mirror on it.
They’ve invented the Pull Request mechanism which allows reviewing and easily merging changes across branches and forked repositories.
GitHub has gone far and wide to be easily integrated with many other tools, so this is why it is another reason why people choose it.
In addition, there is also an on premises solution GitHub Entreprise, but it is pretty expensive.
Its main downside is that it is a single solution for a single purpose and it might not cover all your needs. For some people this might be a strength.
GitLab
One of the biggest competitors to GitHub in the hosted space. There are also many open source projects hosted on GitLab.
Its design and functionality are very similar to those of GitHub but it comes bundled with more features. GitLab promotes itself as more complete solution for “the entire DevOps” cycle, more on that below.
They also have a self-hosted version, which has a totally free option. All in all a great package.
Bitbucket
Another very popular hosted solution for Git repositories. Its design has a different approach then GitHub and GitLab, but it is still very easy to use.
Very similar to the two already mentioned and has all the features you might want.
When it was launched it had better pricing plans than GitHub, which was why we chose to use it when moved from Gerrit.
Sometime ago, GitHub just copied the pricing of Bitbucket. So this no more an advantage.
Another plus for BitBucket is that it is made by Atlassian, who also makes other products like Jira, Confluence and Trello. Atlassian are famous about the good integrations between their products and BitBucket is no exception.
If you are already using anything by Atlassian, it makes a lot of sense to use BitBucket, too.
Just like GitHub Enterprise, BitBucket has a self-hosted enterprise version, and it is just as expensive.
AWS CodeCommit
This service by AWS is similar to GitHub but totally private.
However, it is not as popular or as polished as the solutions that you saw so far. Its main strength is that it integrates very well with other AWS services like AWS Simple Notification Service (SNS).
If you are a heavy user of AWS, as many startups and established businesses are, it might make sense for you to use that.
Azure Repos
Just like AWS CodeCommit, it is a very similar service to GitHub, but totatly private and it lacks some polish and features from the solution mentioned earlier.
However, it integrates very well with many Azure services, so if you are all in on Azure it might be a very good option for you.
Automating continuous integration processes
So far, we only looked on how to store your code and the tools you need to do code reviews. Every tool on the list so far, solves those problems in one way or another.
A critical part of continuous integration is testing and automating that testing. However, it is not only running the tests. It is also inspecting, storing and sharing the results with other tools (Slack, Trello and more).
The solutions below all can run tests, and they can also do many more automations.
Jenkins
One of the oldest and maybe the most popular solution on this list. It is only self-hosted
It is surprisingly easy to install. You need to first install Java and then it is a single file to download and run. Then there are a few easy steps to follow.
What’s more it has any feature you might dream of. When it doesn’t it has thousands of plugins which cover everything else.
You can integrate it with GitHub or with the other git solutions.
I’ve used it on many different projects and it has been very easy to setup and maintain. You basically install it on a machine and forget about it. It can report results back to GitHub or to anything else, so that you can see everything at just one place.
However, as any hosted solution is up to you to keep it up to date, backup and keep it secure, as well as install additional machines when you have a large number of developers.
GitLab CI
Arguably the biggest selling point of this solution is that it is extremely bundled with GitLab as its name suggests.
Super easy to setup and use. Even when you are using a self-hosted GitLab, you have GitLab CI readily available to use.
Circle CI
On several projects we’ve used Circle CI extensively. It is easy to setup and they’ve made it as simple as possible to integrate with GitHub and other services.
It is a hosted solution so, you can just start and use it.
Like any other hosted solution, the main issue is to make it work when you need some custom work.
Maybe you need more memory? Or you have some slow tests? Or you need some custom very specific software? Or you need to run many tests in parallel without breaking the bank? These are all cases when a hosted solution might not be perfect.
Travis CI
It is just like Circle CI. A hosted solution, which you can begin to use immediately.
It is very popular with the open source world. Easy to setup and run. Nothing to install.
TeamCity
A self-hosted solution, which also has a free version. Very poplar alternative of Jenkins but commercially supported by its developer company and as a result modern technologies and improvements are very quickly integrated, much quickly then in Jenkins.
This is maybe its biggest advantage.
Azure Pipelines
This is a hosted alternative from the Azure world. Not as polished as some other hosted solutions, but it has some unique advantages.
The biggest one is that it can run testing and delivery simultaneously on Linux, Windows and even Mac!
This should be great for all kinds of desktop apps or tools required to run on any platform. Also perfect for the mobile world when you have apps for Android and iOS.
Another good advantage feature is that it is not limited to the Azure world, you can integrate it with services from AWS or event with self-hosted solutions.
AWS CodePipeline
Despite a similarly sounding name to Azure Pipelines, it is a very different product. It cannot run testing and delivery on its own. It only orchestrates a pipeline/workflow, basically what happens after what event.
You will need another AWS service like AWS CodeBuild to actually run your tests or build your app.
AWS CodeBuild
From my point of view this tool mainly exists to be part of AWS CodePipeline. Probably nobody uses it on its own. It is basically a build agent which executes your code without you worrying on what kind of machines. Of course, it is fully hosted.
Docker
This is rarely listed among CI tools, but I have seen how it can help especially on the automation phase.
Docker makes it very easy and lightweight to setup the environment that you need to run, test and deploy your apps.
You can create a container and then use it for development, then use that same container for testing and production. It will behave identically everywhere. Docker isolates all the resources and anything inside that container.
The greatest thing is that because it is the same exact environment, you can be very confident in your testing that anything that passes will work seamlessly on production, too.
Everybody is in love with Docker so almost all of the tools I mentioned, self-hosted or not, support either natively or by some plugin or extension to use Docker.
Next
Now that you know the tools for CI, your next step is to go and apply it to your development. Don’t try to find which is best. Just pick one that meats your requirements, and then use it. You can easily change the tools that you use if you have to.