DevOps Tools of the TradeDevOps by Edward Moy Tags : devops, devops philosophy, devops tools, docker, jenkins, kubernetes
DevOps is a fast growing trend which has changed how IT solutions are deployed. The outsourcing involved may be carried out by consultants or specialists who will work through each step of a continuous workflow, known as a “systems development lifecycle”. And there are specific DevOps tools to deliver tailored solutions.
Let’s look at the big DevOps picture and then focus on some Tools (including Confluence, Git, Gradle, Jira, Maven, Selenium, Docker, Jenkins, Saltstack, Chef, Kubernetes, Puppet, Ansible, Splunk and Nagios.). A growing universe of tools means more options and also a greater need for DevOps Services Outsourcing to make the right choices to create the right solutions for clients.
DevOps’s philosophy of collaboration and continuous work drives the use of tools:
DevOps is a set of practices that emphasizes the collaboration and communication of both software developers and other information technology (IT) professionals, while automating the process of software delivery and infrastructure changes, its implementation can include the definition of the series of tools used at various stages of the lifecycle;
…there is no one product that can be considered a single DevOps tool.
Instead a collection of tools, potentially from a variety of vendors, are used in one or more stages of the lifecycle.
This workflow, a/k/a systems development lifecycle, is a “toolchain” which incorporates 7 stages or initiatives. DevOps’s continuous and integrated work is made possible by specific tools used within each initiative — which in turn decides the specific tools used.
- Planning – code development and review, source code management tools, code merging
- Creating – continuous integration tools and build status
- Verifying – continuous testing tools that provide quick and timely feedback on business risks
- Packaging – artifact repository, application pre-deployment staging
- Releasing – change management, release approvals, release automation
- Configuring – infrastructure configuration and management, infrastructure as code tools
- Monitoring – applications performance monitoring, end-user experience
A visual for the basic DevOps Toolchain begins with PLAN and the results are inputs for the next stage and this continues until the DEPLOY stage.
To review, lets understand that a “figure 8” workflow defines the stages of a TOOLCHAIN.
A DevOps toolchain is a set or combination of tools that aid in the delivery, development, and management of software applications throughout the systems development life cycle.
In software, a toolchain is the set of programming tools that is used to perform a complex software development task or to create a software product, which is typically another computer program or a set of related programs.
In general, the tools forming a toolchain are executed consecutively so the output or resulting environment state of each tool becomes the input or starting environment for the next one, but the term is also used when referring to a set of related tools that are not necessarily executed consecutively.
Results from the tools used in one stage of a Toolchain’s workflow, is used for the next stage, or “initiative” but not always. But this idea is easy to help understand what goes on DevOps work.
As for the tools, we’ll look at some of the most common and popular tools used. One important thing to keep in mind is that each client engagement may require specific tools for a unique toolchain. For every job there is a tool and the same idea applies in DevOps.
DevOps philosophy and principles (such as continuous integration, automation, collaboration and testing) which drives the systems delivery lifecycle of a DevOps Toolchain informs the choices of the tools used.
There are many tools available and they are not confined to one stage. Tools may be used in overlapping fashion for multiple stages in a chain – as the results from one stage are forwarded in a handover as inputs for the next stage.
Here’s a chart (via CA Technologies) to show that many DevOps tools are available.
Here are more simplified charts of how various DevOps tools fit in the Toolchain.
We will focus on some common tools used at various stages of a DevOps toolchain.
Planning and Building
The work begins with planning and defining for requirements, metrics, and security. The next step is designing software and its configuration, building, and coding.
Tools may include Confluence, Git, Gradle, Jira and Maven
Confluence is a tool used to keep teams literally on the same page in terms of plans
Git is a very popular “Source Code Management (SCM)” tool which provides version control. Multiple and remote engineers can collaborate, keep track of changes, make backups, move or add files, support nonlinear development, report status, and search for bugs in specific versions. Companies like Microsoft, Amazon and Facebook use Git.
Gradle is a newer tool growing in popularity due to versatility for multiple languages, and faster work via “incremental builds”. It has been adopted by Google for Android Studio.
Jira is one of the most popular planning, project, management and tracking tools used.
Maven is a time-saving automation tool for the building and management, including parallel builds.
Verifying and Testing
At this stage, code quality (i.e. does code conform to the design, does it work as needed and can it be maintained) is the focus. There is testing for performance, security and configuration.
Selenium is used in verifying and software testing, and in keeping with DevOps methods, with automated testing, multiple language scripting and parallel tests. Companies like Google, Salesforce and financial firms use it.
Releasing and Deploying
After the work has been designed, built and tested, the work becomes a “release” that will be “packaged” for deploying. Similar to creating a play for the theater, this phase is also known as “staging” and “pre-production”. The “release” will have a schedule, not unlike a theatrical production, with contingencies for “fallbacks” and “recovery” like stand-in actors for lead actors.
And the “continuous integration” is important here – it is like the choreography of a theatrical show, with all the participants and elements of a production moving simultaneously and smoothly. This approach is at the heart of DevOps’ software development life cycle (SDLC).
Docker is one of the most popular tools which has transformed IT. It’s a platform which helps engineers who want to assemble and build, and run any application anywhere. Docker automates the efficient and scalable deployment of applications with “containers”, specific bundles of an application with libraries and configurations, so that it can work on different platforms regardless of customized settings in different machines.
The result is that more apps runs on less hardware in multiple environments – attractive for both developers and clients. All kinds of businesses like Paypal and Uber use it.
Another release tool includes Jenkins which is a popular and easy to use tool for continuous integration of DevOps stages, with 1000 plug-ins, for building, testing, reporting for debugging, and deploying applications for multiple platforms. Developers can use the tool to build and test software continuous so that changes can be integrated and users can get fresh builds a/k/a updated versions. Companies like LinkedIn use Jenkins.
Saltstack is python-based open source software with an “Infrastructure as Code” design ideal for software-centric data=centers — for configuration management, automation, remote execution, and security.
Configuring and Operating
This stage is more operations-oriented (the “Ops” in DevOps) after a release has been successfully deployed. The work is about configuration and infrastructure provisioning.
Tools for this post-deployment phase include Chef, Kubernetes, Puppet and Ansible.
Chef is an open source configuration management tool to enforce consistency of environment for actual and virtual machines. Written in Ruby and Erlang, it’s popular with developers for streamlining server provisioning, configurations and maintenance in Cloud environments — including Amazon EC2, Azure, Google Cloud, Internap, OpenStack, Oracle and Rackspace.
Kubernetes is a tool, that was designed by Google (originally written in C++ but later Go) for automating deployment, scaling and management. It’s a “container centric” tool which can work with tools like Docker. Various cloud providers offer platforms based on Kubernetes or an “Infrastructure as a Service (IaaS) environment on which it can be used.
Puppet is another configuration management tool, which also runs on both unix-like and Microsoft systems. It is used for automation of processes, including software inspection. provisioning, patching, and management of operating system and application components across enterprise and cloud environments.
Ansible, an increasingly popular tool, acquired by Red Hat, which simplifies the automation of the entire software development life cycle (SDLC), including the automation of redundant manual tasks, configuration management for system consistency, and deployment of applications. The basis for its popularity is that its design, in contrast with other tools including Chef and Puppet, is its minimal agentless architecture — orchestration doesn’t occur within a single machine in control.
At this stage the release has been deployed, and has been configured for operation. The work is focused on watching and measuring how the solution is working – monitoring for end-user issues and experience and IT infrastructure performance. The tools used focus on performance testing and test automation.
Monitoring tools include Splunk which provides visualized analysis of system performance and troubleshooting. Developers can search and solve for errors during the development and testing phases. Another common tool is Nagios which provides comprehensive monitoring of systems, servers, networks and infrastructure.
This process starts all over again in a continuous software development life cycle loop of design, creation, release, testing and monitoring.
There is an abundance of DevOps tools and DevOps specialists can be invaluable in making the right choices for the right solutions.
Here’s a chart from Xenia Labs as a reminder of the depth of tools available in this field:
OSOLABS is one phone call or email away from helping clients with DevOps services.