How computers work? - Part 1: Information and computation

In our current world it is a given that you had an incredible amount of exposure to a plethora of different types of computers. They can be found everywhere and in almost are put into almost everything - cars, home appliance, military equipment, consumer electronics and much more.

So did you ever wonder how does a computer work? Or even a more basic question - what actually is a "computer"? No matter if you're a complete layman that simply took an interest in the subject on a whim or an aspiring young software engineer who hopes to hone his craft with this newfound knowledge - for either I will try to present the subject in an approachable manner so that at the end you will understand what is a computer and how it operates on a elementary level.

The subject is a big one, even when trying to simplify different parts of it, thus it will be released as a series of articles to be read and understood one after the other.

Information

The fundamental idea that we need to start our journey is one of information - this is the elementary building block from which more advanced concepts will be constructed upon and understanding it, at least at a basic level, is essential to proceed further.

So what is "information" then? Everybody has an almost instinctual sense of what it means but when asked to define it they might find it quite difficult. This is understandable as it is a quite abstract concept - in a way everything that surrounds us from the smallest particle to the Universe as whole can be defined as some kind of information, thus finding a "common denominator" might prove difficult as it needs to apply to such a broad spectrum of "stuff".

There are a few definitions, many can be easily found all around the Internet, but for us let's focus on the simplest form that I think portrays the idea behind the concept well and is based on measurement:

Information is measured by the number of unique configurations of a system.

In essence the question about information within a system is - how many configurations does a system have which I can differentiate between. Of course here the question arises about our ability to measure those states and configurations - thus the effective/usable information content of a system is bound by our ability to observe it and make distinctions. It doesn't matter if a system has a million different unique states if we cannot observe them all and differentiate between them.

Information with a well-defined discrete structure is called data. All "practical" computational systems deal with data in some form or another but we'll focus in the first part on the more general concept.

So maybe let us look at some basic examples of systems to try to internalize the concept of information a bit more. Let us have a:

• Coin
• Standard six-sided dice
• List of 10 names on a whiteboard, each name is taken from a set of 5 unique names.

As explained above to measure the informational capacity of a system we need to first know how many unique states can it be in. For the coin and dice that is pretty simple but the third example is the interesting one:

• Coin - we have two possible states, head and tails (H, T)
• Dice - pretty simple, six states marked by number from 1 to 6 (1, 2, 3, 4, 5, 6)
• Whiteboard with names - the most obvious answer would be 50 as there are 10 possible rows ("slots") for a given name and each has 5 possible values. But what about repeating names? If we have "Henry" twice doesn't that mean it is the same thing? Well, no, not really. As each names has an additional piece of information attached to it - the row in which it is in. So if each name takes one row we can have states like (Henry, 1), (Henry, 2), ... and each of those will be unique!

Hopefully you've managed to gain some insight into the concept but if you're still unsure about the specifics that is also fine. In the next section we'll delve into a concept that is deeply tied with information which is equivalency between two systems or pieces of information.

Why cat is equal to a coconut

The next concept I would like to explore here is informational equivalency and encoding/decoding of information which will also be crucial to understanding e.g. how a computer can actually draw anything with those 0s and 1s that are in its memory.

In short informational equivalency we'll define as:

If two or more systems have the same amount of unique states and there exists a mapping between them then they are equivalent in regards to information.

What does the above exactly mean? Well let's start with a simple example - a two-sided (mathematical) coin.

When we throw a coin what we're interested is one thing - has it fell down heads or tails. But what is exactly here "heads" or "tails"? Is it a portrayal of a president of a country? Maybe some landmark important to a given country? Well the truth is that doesn't matter - for our purpose (game of coin throw) all of those are equivalent, given some initial agreement as to which value is the tails and which one the heads.

So we can have 3 possible coins like (H - Heads, T - Tails):

1. H: Flower, T: Tree
2. H: King, T: Queen
3. H: Sun, T: Jupiter

And the above coins are equivalent to one another in our context which is the game of throwing a coin. This is because the rules of the game are not interested what exactly is on a given coin - thus we implicitly have a mapping like:

• Flower -> King, King -> Sun, Sun -> Flower
• Tree -> Queen, Queen -> Jupiter, Jupiter -> Tree

We now have a "recipe" to go from each Head/Tail specific value to another one (we can also chain those together, compose them). So we can freely "transpose" the abstract state between each coin and thus for our purpose they are effectively the same thing - just different sides of the same (abstract) coin so to speak.

The important thing here is that the mapping(s) is/are isomorphic (1:1) and that by applying them we can always go back - we do not loose information by doing so. We're simply changing it's form to another one.

Why is the above important? Well in general there are situation when a given form of information is easier for us to operate on than the original one. Of course our result should be always the same as when doing it one the information in the original form thus the requirement for an 1:1 mapping of the states between the two forms so that we:

1. Transform the information to our preferred form
2. Construct the result based on the information in our form
3. Transform back the obtained result to the original form

An example would be having the task of summing up the results of 5 throws of a six-sided dice. But we don't like small numbers thus we use a mapping that transforms each side to its 10x multiple so that: 1 -> 10, 2 -> 20, etc.

When we sum up the results in our form we simply divide it by 10 and we get the result in the original form. Nothing lost and we obtain the correct result.

The concept shown above is one of encoding and decoding - the transformation of the information into a different form which is more tractable in another context. Here we explored more precisely lossless encoding/decoding.

Computation

So we have all this information we've talked about earlier, now what? Well usually we would want to get something out of it - to transform it so that we can get a new piece information that is useful to us.

We've talked about this briefly in the previous section where we encoded and decoded information from one form to another by applying some kind of mapping between the two forms. We will now explore this subject further.

Some examples of transforming some piece of information into another one that is more useful:

• Calculate 2+2
• For 1, 2, 3 get the corresponding alphabet character
• From the text "a cat is a cat" get the number of times the word "cat" has appeared

In short we would want to apply a well-defined mapping (which can be composed of other mappings) from the given piece of information to another. For the above that would be:

• Calculate 2+2 -> 4
• For 1, 2, 3 get the corresponding alphabet character -> [a, b, c, d]
• From the text "a cat is a cat" get the number of times the word "cat" has appeared -> 2

In the above we've applied some kind of mapping to transform a piece of information into a result that we wanted. And that is essentially a computation.

So let us define it as:

A computation is the result of applying a well-defined mapping to a piece of information to obtain a result.

Why "well-defined" you may ask? Well we need to know exactly what to do with the information we get to obtain the result - thus it should be well-defined.

In the above examples we can discern a certain pattern - the information transformed contains two essential pieces:

• What should we do - the instruction
• On what information should the above instruction be applied to - the input

So for our examples we have:

In the above we actually already applied a transformation upon the information given to us giving it structure - as mentioned in the beginning we call that data. As we can see the process of transforming chaotic information by pushing it into a well-defined structure to create data is also a type of computation - and as a side note in most cases it is the first one that a practical computer system makes to e.g. make sense of our hand movements and gestures on the touch screen or mouse.

There is also an important type of mapping that we should mention here - one which consists of a finite sequence of rigorous instructions. We call that an algorithm and is the core concept upon which will constantly reappear in further parts of this series.

What's next?

So now we should know a bit what is information, data, encoding/decoding of information and computation. If the above might be a bit dry it is essential to understand these concept at least on a basic level to truly understand how computers work - as a computer is simply a machine to perform those computations with inputs and instructions that we provide it with.

In the next part I plan to expand upon the theory with defining a basic theoretical model of a machine with which we could perform those computations - the Turning machine along with the Von Neumann architecture as the core theoretical components that make up the modern computer.

Also the topic of information and computation is much broader than just the more practical angle that we want to explore in this series. It is an extremely important topic within many areas of research in math, physics, biology, philosophy and many others. I would recommend to explore it further if it caught your interest.

If you have any suggestion on how to improve this piece or what should be found in the next one(s) please just reach out to me - will be really happy to hear out opinions of other how I and my writing could improve.

Thank you for reading and until next time!