My Engineering Philosophy

Software engineering has completely exploded in the 21st century. As a result there are infinite opinions on how to approach it, what works and what doesn’t.

Although my opinions are constantly shifting from learning and experience, I thought I’d lay out where I’m at now after 20 years of making software in the hopes it might help you see a different perspective (or help me change mine!).

Engineers must be involved in the earliest stages of ideation

Some organisations might be hesitant to involve engineering at the ideation and feature planning stage. This often ends up costing the org more time and money when engineers spin their wheels on incredibly difficult work that doesn’t need to be incredibly difficult.

Having an engineer involved at the early stages can help non-technical staff understand when two seemingly similar ideas can have dramatically different costs in terms of time and complexity.

Similarly, if you’re in an environment where you’re simply handed work, it’s your responsibility to speak up when you see unnecessary complexity in the task you were assigned. It can be easy to assume that a person senior to you knows best, but if you’re in the weeds building the thing you’ll have context that others don’t.

Time spent simplifying up front saves an order of magnitude in engineering time

I was really fortunate to have a great mentor in my first real job, who constantly pushed us to simplify. When we thought we had a good solution he would tell us no, it needs more thought. We’d spend hours together on whiteboards trying to remove, cut, and simplify the problem itself and then simplifying the solution. Every single time, it paid off immensely as we spent far less time actually building the thing. Rich Hickey talks about this in Hammock Driven Development.

When you’re building something, your assumptions are going to be constantly challenged, and complexity will creep in as you add special cases to deal with the complexity. Once those assumptions and special cases are built in to the system, they become much harder to remove (both literally and conceptually). Simplifying the problem up front will cut out large chunks of those special cases, although the process must be ongoing.

This doesn’t just apply to engineering work. It starts at the early stages of user interface design and ideation. Sometimes what’s thought to be an engineering problem can be solved with a user interface change that better reflects the underlying model and doesn’t fight the system.

Engineering planning should never be about finding the most detailed solution, it should be about finding the simplest solution that solves the problem.

Tech-debt is insidious because it’s hard to see

Having a large amount of tech-debt is like living in a messy home without eyesight. You can’t see the problem until you start trying to move around and do things. The more mess there is, the more likely you are to trip over at every step, or break something critical.

Worse, imagine you have a project manager who never has to step foot in the house telling you “We don’t have time to clean up, just get it done!“. It’s clear to you that things will get done much faster if you could just invest a bit of time tidying up, but it’s hard to convince someone who can’t see the mess.

The problem here is a lack of visibility. As an engineer it’s easy to see where tech-debt has accumulated, because you’re the one constantly tripping over the mess and going home with bruises. In these situations it’s up to us as engineers to explain the problem, and the benefits of spending time on cleaning up (and yes, there are real business benefits to cleaning up the mess).

To clarify, cleaning up doesn’t just mean making the code pretty. It’s about making the code clear and understandable, which comes from simplifying things and making assumptions explicit.

Some people argue that code quality doesn’t matter, only business value matters and customers never see the code. Sure that’s true if you’re only looking at the short term, but over the long term that debt is likely to cost the business much more in terms of development speed and agility.

But why does tech-debt accumulate in the first place?

Tech-debt is an unavoidable result of changing assumptions

As a project grows, the assumptions that were made yesterday are often invalidated. When these assumptions are invalidated, we have a choice. Do we rewrite parts of the application to account for the new assumptions? Or do we hack something in to just make it work, and leave two competing sets of assumptions in the code?

The hack approach is almost always faster, but it results in dissonant software. Engineers who join the team are confused as to why there are two (or ten) competing assumptions in the code. Engineers who know the code well are confused too after not seeing the module for a while. Eventually, the knowledge and understanding no longer lives in the system, but every engineer on the team has to hold a set of competing assumptions and special cases in their own head to match those in the system.

This is where the boy-scout rule comes in.

Always leave the campground cleaner than you found it.

Avoid the temptation to do the quick hack. You might be under pressure, but if you don’t push back all you’re doing is making life harder for other engineers or your future self (with the obvious exception of truly time-critical things, in which case use your judgement and prioritise cleaning it up later).

Small improvements are more successful than large ones

In my experience the larger a milestone or goal is, the more likely it is to fail. This doesn’t mean large projects can’t be done, but even when building a very large system it’s typically best to break it down to smaller achievable chunks.

I think the problem here is the limitations of our own minds. Computers can hold a whole lot of information, but our brains can only really comprehend what we’re working on directly.

When a project or goal is huge, there’s such a large space of “stuff to do” that it’s hard to keep it all in your working memory. It can become overwhelming and exhausting. In my opinion a very large part of engineering is working out how to make the code comprehensible to humans, because humans are (for now at least) the only ones who need to change it.

Instead, get something working quickly, and then build on top of it. Don’t be afraid to rewrite your assumptions as you go.

Naming is important

Internal naming of variables, modules, services etc is sometimes mocked as pedantry, but I believe it’s important to get right. Naming not only better explains what’s going on, but it guides the future of that module, what it should include and more importantly what it should exclude.

Names that are too general (eg utils) tend to accumulate a lot of cruft over time, as they become an easy dumping ground for logic. Names should be clear, specific, and obvious.

Prefer explicit over implicit

In general, making assumptions and expectations explicit in the code itself helps immensely in future iteration. This can be done through naming, documentation, static typing or approaches such as Domain Driven Design.

Implicit code and overly succinct logic can look beautiful, but it can also make it much harder for future readers to interpret the meaning behind it.

Static typing improves productivity

Static typing helps with two points I made above:

Making your assumptions explicit
Helping you deal with the complexity of the system

By taking care of a lot of the connections and requirements of your code, a static type system allows you to offload a lot of cognitive power to the compiler. Working with the static type system feels like working with a buddy that’s always watching out for your mistakes. To take things a step further, a good static type system will also allow you to encode your intentions for more correct code.

The big features for a good static typing system for me are:

Explicit nulls. The compiler won’t allow you to accidentally use a value that could possibly be null without first checking that it’s not null.
Algebraic Data Types. The ability to specify types with variants, limiting the state space of possible values (and errors).

Some languages that do support these features at a first-class level are TypeScript (in strict mode), F#, Rust, OCaml, Haskell.

For tiny throwaway projects, engineering quality doesn’t matter a whole lot. But when projects are ongoing, long-term projects that need to change with the winds of business, engineering quality matters a whole lot.

A codebase that’s easy to work in means faster development times, happier staff, less turnover, fewer bugs, and more chance of catching black swans before they appear.

This is not to say we should strive for an unachievable perfection, but nor should we ignore quality just because it can’t always be seen by customers. It’s all about finding the balance.