Software Entropy

Defining Entropy

Entropy is a measure of chaos, or disorder, in a system.

My college physics professor described entropy using two shoe closets.

Imagine a clean shoe closet, where all shoes are paired and sorted by color. The closet’s entropy is the total number of arrangements its shoes can have. A clean closet’s entropy is relatively small. There may be a few pairs of grey or blue shoes that can be switched around – but this doesn’t add much complexity. In a closet with low entropy, it’s easy to add or remove shoes from that closet as needed.

Now imagine a messy shoe closet. None of the shoes are paired, and they’re all tangled in a big pile. How many possible combinations can these shoes be in? You can quickly find out by trying to pull out the pair you want. The messy shoe closet has a much greater entropy than the clean one.

In short, we measure entropy by counting the number of possible states a system can be in. More states mean more entropy.

Entropy in Software

In software, our building blocks are simple enough for us to measure entropy in a crude way. Take this model for example:

  createdAt: String
  buyerId: String,
  sellerId: String
  amount: Int

As simple as it seems, this model is like our messy shoe closet. There are many more ways for this model to be wrong than there are for it to be right. We can see that by comparing it to an organized shoe closet:

  createdAt: DateTime,
  buyerId: UserId,
  sellerId: UserId,
  amount: Price

When `createdAt` was an arbitrary string, it could take on invalid values “foo” and “bar” just as easily as a valid value “06-23-2020”. There are many more possible states that the field can be in, and most of them are invalid. This choice of a broad data type allows chaos into our model. This unwanted chaos leads to misunderstandings, bugs, and wasted energy.

When each model is strongly typed to a strict set of values, this chaos is minimized. DateTime, UserId, and Price are typed such that all possible values are valid. Accordingly, these types are more predictable, easier to manipulate, and lead to less surprises in practice.

As in life, entropy is not all bad – some of it is desirable and some of it is not. In software, we need entropy to a certain extent: our code is valuable because it supports a variety of possible dates, users, and prices. But when this chaos grows beyond the value it adds, our software becomes painful to use and painful to maintain.

Modeling Software Entropy

Given our observations, we can describe a simple rule:

complexity = number of total possible states

A construct with only a few possible states is simple. Booleans and enums are much simpler than strings. A system with one moving piece is much simpler than a system with many moving pieces.

Sometimes, our problems are essentially complex. In these cases, our solutions need some essential complexity to match. But when does essential complexity become unnecessary? In these cases, we can use another rule:

cleanliness = number of valid possible states / number of total possible states

If there are thousands of total possible states, but only two of them are valid: it’s a messy solution. A simple example of this is representing a boolean value as a string.

if value == "true": do this
else if value == "false": do that
else: throw error

There are many ways for this code to go wrong; not just in execution but also in interpretation. Keeping our solutions clean improves correctness, readability, and maintainability. It’s one of the primary measures of “quality” in my view.

Minimizing Software Entropy

Given these definitions, we can ask ourselves some questions to guide our software decisions:

  1. How many possible states does this solution have?
  2. How many of those states are invalid?
  3. Is there any way to make the solution simpler, by trimming the number of total possible states?
  4. Is there any way to make the solution cleaner, by trimming the number of invalid possible states?

The power of this concept is that it smoothly scales up and down the ladder of abstraction. It applies to basic data types just as well as it does to solution architecture and product development.

How many moving pieces does our solution need? When an unimaginable requirement flies in and tries to blow our solution to the ground, how many pieces can be left standing? When an unexpected input arrives, do invalid states propagate across the system, or are they contained and eliminated on sight? In short, how clean is our solution?

To make life possible, we utilize chaos by creating complex systems that support a diversity of people and their use cases. To make life predictable, we combat undesirable chaos by keeping those systems as clean and orderly as possible.

In software, we work in a world where chaos is measurable and cleanliness is achievable. We just need the right set of signals and responses to make it happen.

Success! You're on the list.

Notes on “Productivity” by Sam Altman

Here’s the original article on Sam’s blog.

“Compound growth gets discussed as a financial concept, but it works in careers as well, and it is magic. A small productivity gain, compounded over 50 years, is worth a lot.”

What are your productivity gains?

  • make a list of what you need to do in a day
  • cut out distractions and focus for as long as you can on tasks that need it
  • break tasks down into the smallest doable chunks
  • arrange the doable chunks into a compelling picture for a project

What you work on

“Picking the right thing to work on is the most important element of productivity and usually almost ignored.”

“I make sure to leave enough time in my schedule to think about what to work on.”

“I learned that I can’t be very productive working on things I don’t care about or don’t like.”

“Everyone else is also most productive when they’re doing what they like, so do what you’d want other people to do for you [to maximize their productivity]. Try to figure out who likes (and is good at) doing what, and delegate that way”

When you work with someone, ask them “what do you like to work on?” You can’t answer this question for a lot of people you currently and previously worked with. You only have ideas, that could be very wrong.

You can go even deeper on this. Before you start a project, understand what each team member’s interests are. Then craft the project with the qualities that maximize your team’s interest. Building something is deeper than just “here are the requirements, build them”. The second- and third-order qualities of a product and the team that builds it determine their success in the long run. Rather than just “what product do we want to build?” – “what kind of product do we want to build?” – and rather than “what team do we want to build?” – “what kind of team do we want to build?”. There needs to be a balance between focus on the first-order results and focus on the second-order results. Trying to milk out as much first-order results as you can in the short term leaves you without a healthy team in the end, and then without a healthy product.

What do you do if you and your coworker like to work on the same stuff? Collaborate and share. Sometimes you get the fun thing, sometimes they do. Make it explicit that you’re sharing – don’t make it implicitly competitive. Learn and teach. Pitch joint ventures you can deliver together.

What do you do if you and your coworker like to work on different stuff? That’s easier, split the work that way. One problem may be that then both of you think that the stuff you like to do is the most important stuff, and that is an interesting problem. Ideally, both of you understand that you complement each other and fill in for each other’s blind spots. Without that balance and appreciation, problems arise.

“If you find yourself not liking what you’re doing for a long period of time, seriously consider a major job change.” short-term burnout should be resolvable by some time off, otherwise there’s a deeper problem.

“It’s important to learn that you can learn anything you want, and that you can get better quickly.” This is along the lines of a keystone achievement. Some achievements are breakthroughs in what you believe is possible and open up a whole world of possibilities for life. The thing about keystone achievements you’ve seen is that they’re hard to foresee, they come accidentally as the result of overcoming some first-order difficulty.

“Try to be around smart, productive, happy, and positive people that don’t belittle your ambitions. I love being around people who push me and inspire me to be better. To the degree you’re able to, avoid the opposite kind of people.”

“You have to both pick the right problem and do the work. There aren’t many shortcuts.”


“My system has three pillars: get the important shit done, don’t waste time on stupid shit, and make a lot of lists”

“I make lists of what I want to accomplish each year, each month, and each day.” You tried doing this monthly, and only the daily ended up sticking. But this was a 10x improvement in my productivity. Maybe i should try for the monthly again.

“Lists are very focusing, and they help me with multitasking because I don’t have to keep as much in my head. If I’m not in the mood for some particular task, I can always find something else I’m excited to do.” You find this too – as long as you wrote down the task, you will come back to the list and it won’t get lost. That trust is critical in a complex and distracting environment.

“I try to prioritize in a way that generates momentum. The more I get done, the better I feel, and then the more I get done. I like to start and end each day with something I can really make progress on.” The power of positive feedback loops – start your day with the smallest positive feedback loop you can build.

“I am relentless about getting my most important projects done” Imagine the kind of life this statement of self-identification produces.

“I find the best meetings are scheduled for 15-20 minutes, or 2 hours.” This is great.

“I have different times of the day I try to use for different kinds of work. The first few hours of the morning are definitely my most productive time of the day. I try to do meetings in the afternoon. I take a break or switch tasks whenever I feel my attention starting to fade” – you used to focus best in the evenings (because you had trouble shutting out distractions), now you focus best in the mornings and afternoons. I’m not sure exactly yet.

“I don’t think most people value their time enough – I am surprised by the number of people making $100/hr that will spend a couple hours doing something to save them $20”

“productivity porn – chasing productivity for its own sake isn’t helpful” the diminishing returns of recursion

“Sleep seems to be the most important physical factor in productivity for me.” You’ve learned this the hard way as well. And not just in pure performance, but in other factors like emotional stability and enjoyment of work.

“great mattress makes a huge difference. Not eating a lot before sleep helps. Not drinking alcohol helps a lot.”

“I use a full spectrum LED light most mornings for about 10-15 minutes. If you try nothing else on here, this is the thing I’d try.” recommends this one

“Exercise is probably the second most important physical factor” – you see this too, mostly in second order effects

“Eating lots of sguar is the thing that makes me feel worst [and thus least productive]. I don’t have much willpower with sweets, so I mostly just try to keep junk food out of the house” – same with second order effects, but also pure performance in terms of focus

“Here’s what I like in a workspace: natural light, quiet, knowing that I won’t be interrupted if I don’t want to be, long blocks of time, and being comfortable and relaxed”

“Like most people, I sometimes go through periods of aw eek or two where I have just no motivaction to do anything”

“In general, I think it’s good to overcommit a little bit. I find that I generally get done what I take on, and if I have a little too much to do it makes me more efficient at everything.” You’ve learned this recently. Being efficient is the critical skill – valuing your time and learning to earn multiples money for the same amount of time.

“Finally, to repeat one more time: productivity in the wrong direction isn’t worth anything at all. Think more about what you work on.” Also from the four-hour work-week – “doing the wrong thing perfectly doesn’t make it the right thing”. Some of the best advice on productivity.