The bastardization of DevOps

Just like "cloud", "DevOps" has become the go-to term for upending technical debt and inefficient processes. That is unfortunate.

Has your leadership told you, "We are going to move away from silos towards DevOps"? Did they explain why? "Faster" isn't a reason. What does that even mean? 

How does it happen?

Over time, companies develop written and unwritten process and tack on technical debt to those processes. Before long, it takes (what seems like) forever to get something done. Leadeship typically starts by adjusting some project intake processes with the PMO. That will invitably fail because the two things leadeship won't do is stop accepting new projects or de-prioritize others.

Then, a big (insert revenue generating) project lands and leadeship dedicates resources to it. They usually put the best and brightest on it, give them cart blanche on decision making and the project sails through in record time. "Why can't all of our projects be like this?" leadership asks, and Flex teams are born. But flex teams aren't cool any more, and some overpaid person decides they will be called DevOps teams.

What is DevOps?

DevOps is actually useful and a natural evolution of todays technology stack. In the olden times, you know 6 years ago, enterprises had data centers and teams dedicated to operations, patch power ping, development and security (we'll get to that later). In those days, the operations teams handled the infrastructure and development teams did whatever they wanted on their staked claim in the wild west. When they were ready to go live, they put their huge buckets o' sundries in the ops teams wagon. After that, ops took care of the wagon and fed and watered the horses. If the sundries spoiled, that was the development teams problem.

Fast forward those 6 whole years to what we have today. Forget the wagon, developers can put individual sundries in their pocket and jump diretly on the horse! Ok, enough with the metaphors...we have virtual servers, containers, software defined networking, automated deployments and data center as a service providers. The development teams can now deploy their apps in conjunction with operational configs and ensure the entire stack operates as expected. This has shifted line between Dev and Ops, but contrary to what most Development engineers will preach to you - it has not removed the line.

Hold the line

Let's be honest, developers are hired to do one thing, get their app out. Regardless of their background or intent, their job is to translate a revenue generating business idea into code. Period.

Developers will use everything they can google and install. At one company, I was told by a developer that I gave security requirements to that my requirements didn't matter,  they were going to do whatever they wanted and it was my job to catch them. You need to understant that is the mindset and you must hold the line as a security and operations professional. Ideally, the separation between development and operations continues to exist, just in a different form.

My precious!

In an enterprise, no single group should have control over an entire stack. By stack, I mean the base infrastructure, the IaC that abstracts that infrastructure, the security controls around the IaC including code promotion/validation rules for the code, and the application code itself.

Many will try and ride leadership's wave of process improvement and DevOps implementation to gain control of an entire stack. You have to ask yourself, "Why?"

Separation of duty is more important than ever and DevOps, as idealized, actually creates more opportunities for separation of duty. Creating a DevOps team does not mean simply putting Dev and Ops people in a meeting and telling them to deliver.  It means creating an environment where Development and Operational Engineers work together during an entire lifecycle. That should mean that each bring their knowledge together to deliver the product. The developer may need to add a library to a container image that is not there by default, the pps person may need to add that library, which may have a process of security validation behind it. The ops person knows the ask is coming and begins to prep for it. Conversely, the ops person may let the developer know a certain open source library does not meet licensing requirements and the developer works with ops on another alternative.

This doesn't give either team complete control of the process, it removes barriers between them to speed up the process. In the end, the ops team may expose an API for the developers to deploy their applications into. This API should have SLAs and the ops team should provide documentation and contracts to the developers. The code implementing the API layer should be in a separate stack, especially if it implements security. The developers application code will likely be in it's own stack, possibly utilizing a completely different set of tools. As long as the two can integrate, this would be a perfect implementation of DevOps. During the lifecycle, dev and ops can work to tweak their own code to satisfy requirements.

It's not about the tools

Development needs and operational needs are different. People should quit trying to make them the same. In an ideal world everything would be implemented "as code" but that is just not the case. Operations and security infrastrcuture sometimes cannot be implemented as code. There is still some manual configuration and hans-on processes required to get things to an automatable state. That's normal.

DevOps is not about the tools, it's about the people and the integrations between the teams. Development managers know, their teams' change tools like the wind. Something new shows up at a conference and the next thing you know, your deployment automation stack moved from product A to product B. Infrastructure doesnt work that way.

A data center, virtual or physical, has finite capabilities. Assume your data center has Juniper networking gear, Palo Alto firewalls, and VMware virtualization - with PKS (after the recent acquisition). Your company invested millions in all of this gear and training and the staff to run it. As an ops team, what you can automate depends on the sum capabilities of the prodcuts on the floor and the tools you choose. Change either and your capabilities change.

Focus on capabilities, contracts

Instead of focusing on specific tooling, spend most of your energy writing contracts between your dev and ops teams. This translates into development and ops teams sharing what they need, how fast they need it, what their cycle looks like and the integration capabilities of their tools. Identify gaps between exising tools and SLAs and have the teams address them. Once an agreement is reached, document the tool capabilities/requirements and SLAs and have leadership approve the contract.

The contracts provides a set of known parameters, or "givens", for the teams to base their tooling choices and interactions off of. It also removes the constant tooling discussions that end up derailing progress.

Evangelize the true meaning

DevOps, DevSecOps, SecDevOps are neither a good nor bad idea. They are simply a result of current automation capabilities and the push to increase speed to market.

Don't let an overzealous operations or development leader bastardize the true intent and benefit of DevOps for their personal kingdom building. Remember, in an enterpise, no single group is supposed to have complete control of all stacks or all code. Anyone trying to sell that idea has an ulterior motive and is up to no good. Teams should be free to create their own stacks with their own tooling, as long as it meets corporate security and API integration standards.

Shift your security and operation teams left. Create an IaC abstraction layer for DevOps teams to consume that bakes security and operational baselines into automated evelopment deployments. The ops people on the DevOps teams should focus on improving the abstraction layer to meet development needs.