Why DevOps will not help you? ... and what to do about it.

"DevOps will solve all your problems!" - How many times have you heard that? Let me guess - too many times. And you're probably thinking, "Yeah, right... but I'm working on embedded systems here!"

I've been there. After spending years in both small embedded projects and large-scale DevOps operations, I can tell you this - DevOps might not help you, at least not in its traditional form. But before we dismiss it entirely, let's examine what it was actually designed to solve (and why that matters for us embedded folks).

What problems does DevOps actually solve?

You see, DevOps emerged from web-centric software projects where rapid updates and quick deployments are crucial. Think of SaaS platforms, web systems, and mobile apps.

• Projects that can release multiple times a day (sounds crazy, right?).
• Large-scale projects with multiple teams working on a single product (we're talking hundreds of developers).
• Complex environments requiring consistent automation (because humans can't handle that scale).
• Organizations dealing with complex IT infrastructures (cloud this, cloud that...).

The core problem it solves? The traditional divide between Development and Operations teams. In classic software development, these teams work in isolation:

• Developers write code.
• Operations deploy and maintain it.
• No one takes full responsibility for the end product.

This separation leads to:

• Lengthy release cycles.
• Slow response to business changes.
• Blame games when things go wrong.

DevOps tackles these issues through:

• Team integration and shared product ownership.
• Automated processes via CI/CD pipelines.
• Agile feedback loops for continuous improvement.

But here's the thing - your embedded systems project probably looks nothing like this. Or does it?

Are our problems really that different?

You might be thinking: "That's great, but we're building embedded systems here. Physical products with hardware, firmware, and supporting software. How does any of this apply to us?"

• "We don't need Operations - we're building a product that hasn't even been released yet. We'll ship it when it's done."
• "We don't have organizational silos. Our entire development is handled by a single department."
• "Our product is hardware-based. We can't just push updates multiple times per day!"

Fair points. However, let's take a closer look at your development process to identify potential problems.

• Sure, you might have one team (or even work solo), but you still have distinct roles: Hardware Engineer, Firmware Developer, Software Developer, and Test Engineer. Whether these roles are filled by different people or you're wearing multiple hats - poor project management can still create internal silos, communication barriers, and blame games.

• Your project files move between computers in different versions. Each team member uses different tools and environment configurations. This makes it impossible to ensure consistency in your builds and releases. "But it works on my computer!" - well, can we ship your computer to the client?

• The traditional waterfall approach: hardware first, then firmware, then software, with testing at the end. This sequential development significantly increases project risk. By the time integration happens, it's often too late to change fundamental assumptions.

Where you should focus your attention?

Let's talk about what keeps embedded engineers awake at night - system integration. This is where DevOps "bring the pain forward" principle becomes your best friend. Instead of waiting until all components (hardware, firmware, hardware drivers, and application code) are "ready" for integration, you tackle these integration challenges from day one.

Set up automated tests for your hardware interfaces early, create integration pipelines that verify your driver compatibility, and catch those nasty cross-component issues before they snowball into project-delaying problems. Trust me, discovering that your UART driver doesn't work with your new bootloader is much better during development than during final system integration!

Let's see how we can adapt DevOps principles to actually solve these challenges.

Let's look at the traditional DevOps cycle depicted in the diagram below.

All of these steps remain important to us. I would just like to slightly modify their names and interpretation.

• Plan - This must stay. A good plan is the most important part of any methodology.

• Code & Build - Let's combine these steps into one to avoid suggesting that we only build software. We are building a physical product, and software is just one element of the project.

• Test - Keeping this as a separate step would suggest that testing happens only at this point. Instead, we should recognize that all other steps include some form of testing. Therefore, let's remove it as a separate step.

• Release & Deploy - In our context, these steps are equivalent to integration. Instead of releasing and deploying software to production, we need to verify that all elements of the project remain compatible with each other.

• Operate & Monitor - In our setup, these steps can be combined into a single feedback loop element.

So, with above assumptions, let's sketch new diagram directed to our needs.

• Plan - Refine and verify your requirements. Focus only on features for the next iteration. Assume everything else will change - plan for change.

• Build - Create the next product increment. This doesn't mean a complete device prototype - just the minimum needed to deliver a new feature or test a hypothesis. If you can't prepare next hardware iteration, consider to simulate or emulate it.

• Integrate - Integrate hardware, firmware and software as often as possible - at least once for every iteration. This helps catch requirement mismatches or gaps early in the process.

• Analyze - Monitor your product's behaviour. Draw conclusions from the collected data. Then, adjust your plan based on new insights. Then, start the next iteration.

• Release - After N iterations, when the product meets requirements and you don't see more room for improvement.

Here's what you'll need to make this work.

First, change your attitude. Instead of looking for excuses why it can't be done because "we're doing embedded here", think about how to make it possible.

Interdisciplinary Teams

Gather Hardware, Firmware, Software, and Test engineers in one place and let them build together - iteratively.

• Build incrementally in parallel across all domains.
• Foster direct communication on a daily basis.

Agile Project Management

Remember: Agile isn't about ceremonies - it's about reducing risk.

• Plan for change: Assume everything could change at every stage of the project, and be prepared for it.
• Implement feedback loops: Ensure your project planning is based on real data, not assumptions.

Automation

Aim to complete the feedback loop as often as possible.

• I'm talking days, not weeks. Ideally, multiple times per day.
• Automate each step in your process to make this achievable.

Centralized Repositories and Managed Environments

You need a single source of truth for your project files and tools configuration.

• Ensure consistent releases by always building and testing from the same centralized project configuration.
• Ensure that each team member uses the same tools and environment configurations.

Not sure where to start?

Start small, focus on what brings value, and scale up only when needed.

Remember, you don't need enterprise-scale tools or a huge team to implement these practices. You can start all of this today, on your laptop, with free tools - no fancy infrastructure required.

If you feel confused, don't worry. In the upcoming publications, I'll show you practical examples of how to implement these practices. Subscribe to the newsletter so you don't miss out.