DevOps: Ultimate HA Environment

Hi there, I’ve been reading a few articles lately about Docker being ready for production environments or not. From what I’ve read so far, there are as many good reasons (and experiences) to say Docker it’s ready, as there are to say the exact opposite (Exhibit A), so I decided to run a test by myself.

The test will consist in two sages. The first one will be to properly create an server host infrastructure to support an scalable HA environment, and, of course, run an environment in HA. The second stage, will be to execute some performance tests over the HA Environment created on the previous stage, and see how well (or bad) it behaves.

Let’s begin with Stage 1, creating an cluster infrastructure using OpenStack and Docker (Machine, Swarm and Networking) with a few nodes, and execute a REST service on it.

Read More

DevOps: OpenStack + Docker Swarm

Hi there, I’d like to write about a big word today: “Scaling”. Working with Docker and creating containers for each service is great and all, but once your environments start to include several containers, or if a single container running the service is simply not enough to handle the workload, you need more resources for your host, or, another way to scale you app. And that’s where Docker Swarm comes to help us. It allow us to create a docker cluster, in which we’ll be able to run several containers in different hosts as if they were just one, in other words, turns a pool of Docker hosts into a single “virtual host”. Also, thanks to its discovery feature, we’ll be also able to add, and remove, nodes dynamically. This way, once the host environment reach its limit, more nodes can be added, and new containers  executed to handle the load. And, as it serves the, well known, Docker API, tools as Docker Compose, or Jenkins can use Swarm to transparently scale to multiple hosts.

Read More

RTC SCM: Merges

It’s pretty common in projects to have to develop features, or fixes, at the same time for the same release. It’s also common (or even natural) to want to have an isolated workspace to work without the interruptions of other team mates changes, and that’s when you create a new stream just for yourself, but of course, at some point, you’ll have to integrate your changes with everybody else’s. Or, maybe you are particularly well organized and don’t need to work on isolated workspaces, but some manager asked to include a feature ( again, or fix), from one version/release into another. In both of these cases, the solution is to merge (Yikes!). Read More

DevOps: Incremental Builds

Hi everyone! I’ve recently been working on improving our Continuous Integration environment. The main idea of the approach is to have the project being compiled, installed and tested (UTCs and Functional Test), each time a change is submitted to the repository.

This sounds really good, isn’t it? We make a change, submit it to the repo, and a few minutes later we receive an email with the results of the impact of our change. Yes, it certainly sounds good, but… actually this is a big but, so, BUT, this has a huge problem. Imagine you have a Maven project, with 2 or 3 submodules, maybe a 100 UTCs, the hole project generates a single war file to be deployed, then sure, how much time can that take to be generated, deployed and tested, 5 minutes, may be 7. Image now a project at a completely different scale, with at least 30 modules, more than 2000 UTCs, and more than 8 war, ear and/or jar files generated. That will take more than 7 minutes to compile and run tests for sure. With a project of that magnitude, you would have to wait almost an hour to know if you broke something, even when your changes only affects a single module.

Here is where the Incremental Builds enters the scene. Wouldn’t be better to compile and test just the modules you’ve changed instead of the hole 30 modules? Definitely yes. Read More

RTC SCM: Flows

As mentioned on a previous post (RTC SCM: Cake or headache?), the IBM guys introduced some new concepts into the SCM world. Another of those is the concept of “flow”. I haven’t mentioned flows before because I don’t consider them part of a basic training (people could get even more confused). But it’s definitely something you’ll have to learn if you insist on working with RTC SCM, so here it is.

Read More

DevOps: From Zero to Production #4

If you have been following the DevOps posts in order, then you may have a few Docker containers running at this point. Good for you!, but, how are you backing up those docker images? As all the servers are running on a Docker containers, then the best option to store them would be a Docker Registry. Now, if you don’t have one of those already, keep reading 😉

Let’s start with the concept. A Docker Registry is a server side application that allows you to store and distribute Docker images. It’s particularly useful for improving the control over your images and where are they being stored, and for integrate the distribution into a pipeline or development workflow.

The easiest option is to create an account in the Docker Hub, the official and public Docker registry. They even offer an Enterprise version for commercial use. But, if what you need is something more private and internal, there is the option of running your own registry instance, and here is how. Read More

DevOps: From Zero To Production #3

As I wrote in the first of this series of posts, there are several tools involved in the Continuous Integration Life Cycle, and, if the Source Control System is the main one, then, the next in the list of priorities is a CI Tool to automate the whole process.

I have to be honest, I can’t be objective here, I truly believe Jenkins is, by far, the best option. It’s a Java based application, that can practically adapt to any kind of project through its plugins. It’s Open Source, and there is a big community supporting the project, which means problems being fixed every week, and new plugins being released constantly. It really worth a shot.

NOTE: For production environments, remember to use the Long-Term Support Releases. I’m using the latest LTSR for this example, version 1.609.1.

Read More

DevOps: From Zero To Production #2

If you are working on a software project, and you already have a source control tool running, then the next thing on the list is a tool for managing the project, an issue tracker.

Altassian has pretty cool tools for that, but if you are into open source, or prefer something more customisable, I recommend Redmine. It’s a really complete and easy to use tool, you can learn more about how to manage you projects with it here: Getting Started with Redmine. If you like it, I can show you how to set your own Redmine instance running on a server, just keep reading.

Read More