Avoiding worry driven development

Sofware dysfunction is more often motivated by anxiety, fear, worry and embarassment than it is by a lack of technical skill. Engineers avoid work that needs doing because they’re afraid of becoming entangled with a nightmarish task, or because they’re afraid of looking stupid, or harming their career by spending time on grungy work. The task itself is rarely that difficult; it just seems so, which is enough of a barrier to discourage anyone from picking it up. One of the highest-leverage things you can do as an engineer is to recognize this emotional reaction in yourself and work to counter it. If you’re working on a single task, doing this can make your implementation significantly cleaner. If you’re a senior engineer with responsibility for a whole system, this can help you address high-impact architectural or operational issues.

You can see this in the common advice to sleep on a problem and come back to it in the morning. A bug that seems impossible when you’re tired and frustrated is far more approachable when you’re well-rested and caffeinated. Tedious investigative tasks (like scanning the logs for a bug or tracing all the possible code paths in a function) are still tedious in the morning, but infinitely more doable. You can knock it out in half an hour while drinking your morning coffee, rather than spending hours the night before avoiding the tedious work and hoping for a flash of inspiration.

Necessary work avoided becomes a haunted forest in the codebase. One engineer’s avoidant emotions turn into dangerous team habits as engineers teach each other to avoid touching certain files, or to pass odd-looking production bugs to the operations team. “This network behaviour is dark magic, let them handle it!” What began as a single engineer’s worry is now a team’s dysfunction, and can eventually metastasize into actively blocking engineers from working on the original technical problems: if you’re avoiding working on a worrying problem, it can feel threatening when someone else comes along and tries to solve it. What if they succeed, or worse still succeed easily? That would reveal that you weren’t good enough to solve the problem.

Just because you’re touching a scary part of the system doesn’t mean you’re facing scary work calmly and rationally. Engineers often approach thorny work like they’re going through a haunted house: either trudging through with eyes half-closed, or rushing through at full speed trying to get out as quickly as possible. Both tactics aim to take in as little of the frightening environment as possible, and neither tactic is a good way of solving real problems in technical systems. If you’ve ever just tweaked setings or frantically moved code around until it worked, you’ve done this.

From the inside, this doesn’t feel like emotionally-driven avoidance. It feels like working on legacy code, sharing friendly complaints about it with other engineers on your team. This is in part because a fear of hard technical problems can be a rational fear. Untangling messy parts of the codebase, identifying and removing dead code, cleaning up build, deploy and monitoring systems - these tasks often feel overwhelming because they can overwhelm you, sucking up days for no real gain. It can harm your career to spend time on this work in an org that mainly values shipping features. (I haven’t seen the other common worry - looking stupid - really be an issue in practice myself.) The trouble comes when you don’t recognise that part of your response is emotional, and therefore overrate the actual difficulty or risk.

How can you distinguish a rational response to a too-difficult task from a learned emotional reaction to a task that is merely worrying? This is a hard problem. I think it is the hardest problem that almost all engineers will regularly face. One approach is to avoid trying to tell the difference at all, and allot a certain amount of time to looking at randomly-chosen neglected parts of the system. Things that reduce stress in general are also good ways to reduce stress and worry about work. Good sleep, eating well, exercise, therapy, and working in an emotionally safe team all have a significant positive effect on the quality of engineering work. Once you start doing this, the difference is usually obvious in hindsight: if the task that worried you for six months took an hour or two of work when you finally mustered up the effort to look at it, you know your worry was unfounded. Dealing with worry is a virtuous cycle. Each difficult task you do makes you less scared of the next one.

Tackling a series of worrying tasks is one of the quickest ways I know to have massive impact in your engineering org. This is because once you understand a thorny piece of code, or a neglected system, you will usually see how to make obvious improvements. In any other part of the system these improvements would have been made already, but since you are the first person to really wrap your head around it in a long time, you get to make them here! Occasionally, you will see the opportunity to completely remove the worrying part of the system, which for me is one of the most satisfying things you can do as a software engineer. Removing things that cause your team stress has compounding benefits to your team, to the systems you work on, and to your engineering org in general.