As the Pendulum Swings

A Headache of Epic Proportions

Recent news about the CrowdStrike Falcon agent that essentially bricked millions of computers worldwide, affecting airlines, hospitals, TV news stations, and countless other businesses, has been chaotic. As a former desktop support technician, I can only imagine the stress and countless hours of cleanup it will take to recover from this mess. Hearing the stories from some of my former coworkers working on the front lines of desktop support has been truly eye-opening.

Reflection on Quality Assurance

This situation has made me reflect on the importance of quality assurance (QA) within today’s organizations. Trends in technology often swing like a pendulum. Traditional software development followed the waterfall model, where engineers designed something, completed their work, and then passed it to the QA team to find bugs or issues. The QA team would either approve the release or send it back to the developers for improvements. This pattern was inefficient, leading the industry to adopt agile software development.

Agile aimed to increase the speed of deliverables by eliminating waste seen in traditional waterfall processes. Instead of having a dedicated QA team that received work post-development, QA engineers began working closely with developers, developing test strategies, implementing standards and tooling, and mentoring developers in quality engineering best practices.

The Value of QA Engineers

As someone who has worked closely with QA engineers and managed them, I have seen firsthand the incredible work these engineers do and the huge benefits they bring to a software engineering organization. Unfortunately, there’s a troubling trend where QA engineers are viewed as either a “cost center” or “non-coders” who consume rather than contribute. This is far from the truth. I have seen QA engineers partner closely with developers to develop comprehensive test strategies that developers alone couldn’t achieve. This collaboration results in shipping reliable and tested software, creating a pleasant experience for customers.

This trend is especially noticeable in the gaming industry. Before I had kids, I was an avid gamer. Back in the N64 and PS1 console days, games came on cartridges or discs and had to be rigorously tested before going to market since there were no opportunities to patch them for bugs post-release. Today, the pendulum has swung the other way. It’s almost expected that games are shipped with numerous bugs, rendering many unplayable. Sure, the game reaches the market faster, but at what cost to customer satisfaction?

Finding a Balance

I wonder about the QA processes behind the CrowdStrike issue. Was there any in-depth QA work done? It’s easy to assume there wasn’t, but I am interested in hearing more about what happened in the coming weeks and months. There are countless examples of broken releases in the industry that required immediate fixes, disrupting customers who rely on us to deliver quality products.

What can be done to address this? Every business is unique, but from my experience in engineering leadership, the pendulum needs to return to the middle. QA should not impede developers’ work, but it also cannot be ignored or assumed to be performed by overburdened software engineers on tight deadlines. We need QA engineers in our organizations. I often use the analogy that in software and IT, we are like “sound engineers at a rock concert.” If we do our job perfectly, nobody knows we are there, but if we mess up, we are the most hated people in the room. This is not a bad thing. Our customers expect the products they pay for to work, and we owe it to them to deliver quality. Quality engineers provide a safety net that protects a company’s reputation and shows customers they are valued.

Conclusion

There are always ways to eliminate waste in an organization, but removing QA engineers and assuming their work can be handled by software engineers is not the answer. QA engineers deserve to be celebrated and heavily utilized. If anything good can come from the CrowdStrike fiasco, it might be the opportunity to have an honest conversation about the need for robust quality engineering practices before shipping something that could have unimaginable impacts.

Until next time, all the best.

-Chris