DevOps is a messy term. It has different meanings to different people, which is a problem. People will look at continuous delivery and think it is DevOps. Some people will see infrastructure automation as DevOps. While others see containers as DevOps, and the list goes on. However, DevOps is about optimizing the development-to-operations value stream along with continually optimizing it. Enterprise DevOps has the same goal, but it has tools to help in large organizations.
With the definition of DevOps above, some would say that is all you need. However, DevOps was originates from startups. These organizations typically have one service or product along with a small organization structure. On the other hand, large enterprises are global in nature with teams spread across the globe and a number of organizations involved in delivering software to production. This is where applying DevOps can fail in a large organization and Enterprise DevOps can help.
Enterprise DevOps tools helps enterprise management understand the benefit of DevOps. It leverages heavily tools and techniques from Lean Manufacturing, which I think is a great fit for DevOps in a large enterprise. However, I am a little biased.
Full disclosure, I am a bit biased towards Enterprise DevOps, since I have experience doing Lean in a manufacturing environment. My lean work was in a dairy, where they took in raw milk and produced store ready milk and cottage cheese. Within in a few months of using Lean, my team and I were able to get rid of $1.7 million of waste from an identified $2 million waste annually. The tools and techniques our team used helped highlight the location of waste within the system. These similar tools are used in Enterprise DevOps.
Enterprise DevOps and Lean
Our team’s champion from my Lean project had a saying about the difference about Lean and Six Sigma, “Six Sigma is great for addressing light wounds, like a band-aid. However, Lean is great when you lost an arm.” I can say from experience, his assessment is true. The tools and techniques my team learned allowed us to quickly problems in the end-to-end process. Beyond just identifying them, we could share them with the line workers and management, and they would see the same problem. This was great for getting buy-in and action to solve the problem.
DevOps requires people to map their value stream, find constraints and continually improve. However, this is fine at a project-level or for a start-up. People can easily see and feel the pain and can take action. However, when you have a large organization with hundreds and in some cases, thousands of software projects (not making it up, one customer did have thousands of project) with a central control for releases and a large amount of dependencies. Then it is hard to find the constraint, especially when you are dealing with multiple internal organizations.
Putting a start-up DevOps engineer in a large global enterprise and asking them transition them to DevOps is a setup for overall failure. For one, they are comfortable at the project-level and could easily optimize a project or two, but not the whole end-to-end process. Two, they would be overwhelmed by the complexity of the organizational structure, software, dependencies and territorial concerns. This is where Lean and its tools and techniques can help deliver DevOps to the enterprise.
How Enterprise DevOps Helps the Enterprise
In a large non-Agile global enterprise, you have a number of groups required to deliver software, which can look something like this:
- Business Analysts
- Development
- Front-end developer
- Server-side developer
- Database developer
- QA/Testers
- Functional tester
- Performance tester
- Operations
- System administrators in non-prod and prod
- Network administrators in non-prod and prod
- DBAs in non-prod and prod
- CCB
- Dependent projects
Based on my experience, I have seen organizations like this, where over-specialization is the norm. Typically, each group is its own organization within the large organization. They then have their own needs and interests. You need tools and techniques to show how business value is delivered and constraints impeding its delivery through the organization.
As mentioned earlier, Lean has a great set of tools for highlighting issues in a business flow. By using these tools, one can create consensus across an organization. This creates the opportunity for change, since the value flow is captured, measured and constraints are highlighted. It is at this point, it is about data and not opinion.
A lot of software delivery processes in large enterprises are built on an evolution of needs over time. In a lot of cases, this processes are not managed in detail, so they become a black box. They may be transparent, but no one knows the true cost of the process and it has not been optimized in years. It does its job, but with the need to deliver software quickly and cheaply, these processes need to be defined, measured, analyzed, improved and controlled (DMAIC). Sounds like Lean is the perfect candidate. Let’s go beyond theory and look at my experience with Lean and how it can apply to Enterprise DevOps.
Value Stream Map
A value stream map is a great tool for showing end-to-end flow of value through an organization as It highlights the key processes and measures. Anyone can easily see where the constraint is in the flow. As mentioned earlier, my team and I used the same tool to get rid of $1.7 million in waste in a dairy.
First, we went out and looked at the process, taking copious notes about the tasks people were doing and measuring what we could. As we looked at all the data we gathered, it became clear these low-level tasks and processes could be binned into higher-level processes. These higher level processes were familiar to the organization, which reduced complexity and increased clarity by using known terminology.
The value stream started with what generated the flow, in this case it was demand from client’s stores. The diary would get demand from the stores, based on their daily sales. This first event in the map showed data like lead time, average request size, and changes to requests to mention a few. Since we were focused on getting rid of waste, the follow-on processes all had waste measured in gallons per day, along with the current price of cost of the waste in US dollars. In addition, we had time in process and between processes, to show chokepoints. Then each process would display measure unique to the process.
It became obvious where follow-on analysis and improvement should take place. So, the team investigated further and used other tools to identify the specific locations where waste was generated.
Supported by Other Lean Tools
At this stage, our value stream map was backed up with the following artifacts:
- Detailed process maps
- Pareto charts
- Cost breakdown diagrams
- Cause and effect diagrams
- Critical success factors charts
- Run charts
These lower level artifacts allowed the team to see where the issue was in the process and to make changes. The value stream map is great for showing the area impacted, but not the specific point in the process. This is where the other Lean tools and techniques help. At this point, you are thinking great story, but how does this apply to office work, we are not in a factory!
Great, But That was a Factory!
If you do anything more then once and you did it similar to the last execution, its a process. If this process is important to the delivery of value to the customer or impacts the delivery of value, then you have a process you need to monitor, improve and control. It does not matter if the process is in a factory or in an office or a ship or airplane, you get the point. A process is a process, no matter its location.
Besides being applied in manufacturing for decades, Lean has been applied to transactional work within offices for decades. It is not hard to imagine your path to production as assembly line. It has stages with names like, QA, UAT, Dev and so on. Within these stages you have a number of tasks to complete. All of this sounds like a repeatable process to me. As it is a process, you should make an effort to continually improve it. This is where Enterprise DevOps and its Lean tools come into help improve an enterprise’s software delivery value stream.
Beyond Lean and On to Automation
Once have your path to production mapped out and analyzed, it is time to look at improving it. Your changes may utilize a mixture automation and process improvement. Remember with technology today, you do not need to put “people in the loop”, when you can put them “on the loop”. Meaning get people away from doing manual processes and let automation do the work, while your people do the analysis and monitoring. Here you will see great benefits.
DevOps and Lean Have the Same End-Game
As mentioned earlier, DevOps is about optimizing your value stream and continually improving it. Lean is about optimizing your value stream and continually improving it. However, Lean has been honed for decades, so it has a rich set of tools and techniques for helping guide DevOps in a large enterprise. This is why I see Enterprise DevOps as DevOps with Lean. For some reason, people forget the continual improvement part of DevOps and Lean.
Today in a number of large enterprises, I see development processes from the 80’s and 90’s. They are waterfall with large teams handing off work from one team to the next. Business analysts creates the requirements, architects create the overall design, developer create the software, and QA tests it before throwing it over the fence to operations. This is a prime example of not continually improving. When you processes look nothing like modern processes, you need to look at improving them.
This is my concern with DevOps, cloud and the other interests in modern technology. I am afraid people will say this is good enough and they will lock processes for another 30 years. In essence, not learning about the true meaning of DevOps and Lean.