Math is Fundamental

As your average American going to high school, I learned the four core subjects (English, Science, Math, History) and a foreign language (Spanish). Since we were allowed to take two years of a subject every year, I ended up taking 5 years of English, 7 years of science, 6 years of history, 5 years of Spanish, and the state minimum of 3 years of math. Believe me, I thought math was useless. It seems like most people agree - the most math they’ll ever do can be computed by a calculator.

And yet, my 16 year old self could not have been more wrong. While I think all of the subjects are useful (I read and write every day, I use the past to inform my decisions, and I lived in a foreign country, so my language acquisition skills turned out to be very important), math and science ended up being much more useful than I thought, not least because I program for a living.

Math turned out to be fundamental.

My first job required accounting skills. I needed to calculate expenses, payroll, taxes, profits, and find out who missed payments for the business. At a small enough scale, you can put this in your brain, but you’ll forget eventually and the business will suffer. Accounting is an important skill. To do that, I used Microsoft excel, but you still need to know how to double-entry bookkeep and measure your revenue, cost to acquire said revenue, and profit. You also need to be able to look to the past, account for any changes due to things outside of your control (employees leaving, seasonal shifts, etc), and control for those to keep the business running smoothly. It takes little more than arithmetic and some plotting, but it makes or breaks many businesses of any size.

Say you have a record which stores an album you listen to. Over time, the record player (and your handling of the record) will dull out the sound of the album – some parts will sound off. The reason? Records are etched with the sound the record player is supposed to play, and over time, the physical needle that the record player uses to read off the record will change those engravings, distorting the sound. Also, if you happen to damage any part of the record, that section of the album will become unplayable, and you’ll have to skip it.

How would we do better? The first guess might be to make two copies of the same data, and then if you damage one, you can go to the other. But that requires a human operator – somebody that can judge that both copies are correct, one is correct, or neither are correct. If we want the machine to be able to do it for us, we’ll need three copies of the data. Unfortunately, that cuts the space on our media by 3, and it still has pitfalls – if you manage to corrupt two of the same parts on two of the copies of the disk, you’ll still lose that chunk of data on disk.

Sounds pretty bad. We can do better – take a CD, for example. If you scratch a small part of the CD, chances are it still works. The reason why it works is because the CD has a scheme for error detection and error correction. The CD is partitioned into two parts: the actual bits and bytes for the data, and digits that can correct the data, should it be damaged. If you damage any part of the CD, as long as the rest of the CD can compensate for it, the CD can restore the bytes it has and play as if there was no problem. This is because of Reed-Solomon encoding, which encodes the data on polynomials and restores the underlying data points. This is also tunable, so you can allocate as many bytes as you want for error detection, so the algorithm can correct fewer or more errors, taking up as little space as required.

One potential scheme involves using 223 data bits for every 256 bits on disk, thereby using up 32 bits for error detection and correction. Since half the bits can be used for correction (16), this scheme would cost (223/256) or about 13% of total disk space, but be able to correct up to 16 bits that were flipped in each 256 bits on disk, or about 6%, or 32 bits that were erased, so 13% of bits.

DNA uses error correcting codes as well. If our DNA wasn’t resilient, we’d most likely have died before we got here.

What about compression? If you’ve tried to compress files on disk, you’ll know that text files are easy to compress, while binary files, video and picture files don’t compress much. The reason why? Entropy. The same entropy from science class. Entropy measures the chaos of a system – in chemistry class, we learn that converting matter to energy, or various energies to other forms incurs a tax – some of the energy is lost, and adds to the chaos of the system we live in. Thus, entropy always increases – if you walk, some amount of kinetic energy spent is lost forever as friction.

The written languages we speak have good compression – we only have a certain set of words and punctuation we can use, and we mix and match them to form sentences. Compression can exploit this fact, and create a dictionary for a file, and turn a file into a set of bits, with the dictionary used to decode the file back to its original format. Compression exploits the redundancy of the underlying data to turn it into a more compact form, thus, it has low entropy. Imagine we used a language with totally random words, which could appear in a totally random way. This language would have high entropy to encode, and thus, would not be compressable. The more predictable an input is, the easier it is to compress, and the less predictable an input is, the harder it is to compress.

This knowledge came in handy when I worked as a chef at a restaurant. Imagine you had a restaurant where you could serve any dish, and customers would come at any time to order any of those dishes. Your input (customer orders) are random across time and space (they can order any dish). Thus, it is impossible to hire a set of chefs, and give them a schedule that would satisfy all of their orders efficiently.

Thus, restaurants have to employ some form of compression. We may choose to focus on a certain cuisine with similar dishes – thus, you limit the set of ingredients you can use to those common to that cuisine. This also allows our chefs to specialize in those dishes, and the customers can only order from a specific set. Thus, we’ve compressed our input in terms of space. But we also want to provide our chefs a good schedule across time – so we may choose a set of working hours (lunch hours and evening hours) that make ordering at certain times impossible, so we can pay our chefs for their most profitable hours without going broke, and our chefs don’t have to work 24 hour days all the time.

So the obvious forms of compression are already done (most places don’t work around the clock, and they don’t serve every dish known to man). But one that came up during my time was how to support delivery. We had to think about how much of the menu to support (some dishes just aren’t deliverable feasibly), what containers we would use to transport it, and how to allocate couriers to deliver the food.

Basically, we had a choice to add entropy to our system in exchange for increased profits. We tried it delivery out for a bit, but found that customers didn’t really enjoy our food when it wasn’t hot off the grill, and our overall orders were coming out slower due to the extra time required to prepare food for delivery, not to mention the decrease in shelf space to hold containers for delivery as well. At the end of the day, we stopped delivering food, and gave up those increased profits, since it wasn’t worth it, in the end. We choose to focus (compress) on what we did best to keep our customers happy.

Entropy also turned out to be fundamental when I worked in finance. We used to say that banks exist to turn risk into reward. We would take our customers money, chase returns, and then the rewards would be split – some goes to the customer, and some goes to the bank. The bank thus has to choose a set of allocations (a portfolio) for the assets that they manage.

They can choose assets that have low correlation with each other (and thus, the portfolio has low entropy), but in exchange, since the risk of this strategy is low, the expected payoff should also similarly be low. They can also choose the inverse, where the assets have high correlation with each other (and thus, high entropy), with high risk, for the chance of high payoffs.

But in reality, you don’t really want to do either. Be too conservative, and you’ll lose all your customers to your competitors who can offer higher returns. Be too risky, and you’ll lose all your customers money. Banks spend most of their time managing risk – calculating the amount of entropy in a particular portfolio, and calculating whether or not it is worth it to invest. The only problem? The risk of financial instruments can’t be seen in advance. It’s always just an estimation. Therefore, understanding entropy and calculating its flows through the market is essential in finance. Do it right, and there’s profits abound. Do it wrong, and you’ll be the next one the tabloids.