Managing complexity


As I become more and more experienced in engineering, the more and more I realize keeping things simple are difficult, especially in terms of design, documentation and communication flows between teams. This is a realization that takes a lot of time to hit hard, but simple systems are easy to maintain while complex are not. This is the core engineering principle that keeps all “complex systems” of the world from spacecrafts to AI systems to even semiconductor manufacturing systems working like magic. These “complex systems” are nothing more than simple systems seamlessly working together to propel various actuators using sensors and software to make decisions. I think it takes true genius to simplify things down to their fundamental truths, understand them bit by bit and solve them bit by bit. More than intelligence, this requires persistence and never giving up. The people (like CTOs for instance) managing these engineering teams across companies are more like generalists who can see the system working from a very high level and derive relationships that a specialist like myself rarely see. This is truly powerful and a kind of superpower that few manage to master. So how do we manage “complexity” and understand complex designs?

First technique I use is keep asking questions, again and again, deeper and deeper. Never stop. People might get annoyed, but a not so well understood system is worse than annoyed people. Because anything can happen when you don’t understand things. A flawed design leading to a plane crash killing 1000s of people? Failure of software systems on spacecrafts worth billions of dollars? One advantage of understanding things deeply is that you also start thinking about failure aspects and think about how we prevent and test the system such that it is not fragile?

Another technique is to slow down and not give in to pressure of delivering systems that you don’t understand. Take your time to truly understand the system at its deepest level. This is a behavioral change that I also find extremely difficult to practice. It requires courage to say a “No” when needed. In the age of agility and delivering, failing fast, this might come across as a traditional and backward method of managing complexity, but when you slow down, you look at the bigger picture, try to connect dots that you did not see before when in a rush and finally understand and master it.

Another thing that helps me simplify things is writing them down and explaining them to myself or a colleague. This is what is known as the Richard Feynman technique. A truly powerful technique. Most of engineering requires writing things down in such a way that anyone new to the team with a basic technical background can understand. Provide clarity when the team feels that there is none. That’s what tech leads do.

Another technique I use is to continuously redesign and improve existing. When you redesign something, you again start from the basics and try to build up on how you would have done it the first time. You take the iterative approach, rethink from the scratch and you simplify things and end up with an improved version in the end. There is always a risk of breaking something, but in the end you learn more about your system than you did before. If something breaks for a reason you did not know, then you truly did not understand it.

These principles and techniques can be applied to any field of engineering. That’s it for today, some thoughts that came to my head when thinking about engineering “complex” systems.


Leave a Reply

Your email address will not be published. Required fields are marked *