Hitchhiker’s Guide to the Blockchain


Blockchain, cryptocurrencies, smart contracts, Web3 and NFT are the talk of the town. Almost everyone has an opinion, but hardly anyone understands the foundations or relationships. The goal of this series, whose first part you are currently reading, is to provide a structured presentation of promises, the technology behind, and the reality. Using everyday analogies such as space-faring, stamp-throwing, anarchist notaries, the concepts are explained in a trilogy in five parts.

(Diesen Artikel gibt es auch auf Deutsch 🇩🇪: Per Anhalter durch die Blockchain.) More on blockchain in the simple, humorous, yet thorough article «Hitchhiker’s Guide to the Blockchain».

Solutions based on blockchain have been proposed as solutions to any problem even only remotely related to IT, digitalization, economy, democracy, and society. Like for every new topic, thinkers, gold diggers, freeloaders, charlatans, and doomsday prophets show up in an colorful mixture, making it hard to keep track of things.

The goal of this series of articles is to provide sound insight both to newcomers and experts, in an easy-to read manner. The focus is — as much as this is possible in this complicated, entangled topic — to present properties and relations transparently, substantively, and in a structured manner. Guidance will be provided by illustrative and memorable, sometimes hilarious, metaphors.

The huge topic will be structured in five layers:

  1. The blockchain proper, a way to store data transparently and immutably (this article);
  2. cryptocurrencies such as Bitcoin and Ethereum built on top of blockchain, which are touted to revolutionize our financial system (in an upcoming article); and
  3. smart contracts, which — as generalizations of (conditional) cryptocurrency transactions — are hailed to simplify building and enforcing contracts (in an upcoming article).
    These smart contracts are used to build:
  4. Non-fungible tokens (NFTs), digital artworks ready to revolutionize artist payments, art trade, and copyright (in an upcoming article); and
  5. the promise of a democratization of the Internet, dubbed Web 3.0 or Web3, also the basis for decentralized autonomous organisations (DAO; in an upcoming article).

Each of these five layers will first be presented and objectively analysed in four chapters, followed by two more chapters of personal assessment and a list of take-away questions. Due to the complexity of the matter, this separation is not always

  1. What hopes and promises stand behind the technology and which problems they are supposed to solve.
  2. An accessible introduction of the inner workings of the actual technology and their possibilities and limitations.
  3. Are blockchain-based solution the only possibility to address the problem? If not, what are alternatives and substitutes?
  4. What is the reality behind the promises?
  5. A subjective evaluation of the technology based on the items above.
  6. A list of questions to ask yourself or an expert, before investing (financially or technologically) into that particular layer.

Ready? Let’s wink our Electronic Thumb and put our Babel fish into our ears, and let’s go! First stop: The foundational layer, the blockchain itself.

I tried to keep this text as neutral as possible. During my research, however, I learned two things:

  1. Blockchain advocates criticize the current economic system as being plagued by greed, inefficiency, lack of transparency, and excessive complexity. However, the goal toward which these advocates work, is a new, now digital, economy, plagued by greed, inefficiency, lack of transparency, and excessive complexity.
  2. Whenever I asked for a tangible benefit of blockchain-based technology compared to a solution without blockchain, the answers have been evading or inconsistent.

Nevertheless, this entire field is due for more insight. The Blockchain ecosystem has raised several important questions, and addressing them is relevant for the future of our society and our interaction with technology. The «blockchain» term was able to score high in the Buzzword Charts for over a decade. That alone should be reason enough for an in-depth analysis and understanding of this phenomenon.

Part 1: The Blockchain itself

TL;DR (or: The summary for all non-managers)

  1. Sufficiently advanced technology is indistinguishable from magic,” but if some technology is being sold as magic, it is important to ask if the technology actually can do what it claims or whether it is only capable of magic smoke-and-mirrors. (Or, to put it another way, if something seems too good to be true, it often is).
  2. If you add blockchain to a crap process, you have a crap blockchain process“. (Or to put it another way: good solutions do not emerge quasi-automatically through the purchase of consultants or technology, certainly not if these bring new complexities and inefficiencies).
  3. A good idea alone is not automatically part of a good solution. (Or to put it another way: even a hammer should get to know other friends than just nails).
  4. Trust increases efficiency and reduces complexity. Blockchain solutions want to do away with any form of trust. They buy this with inefficiency and complexity. (Or to put it another way: complexity is the enemy of security, transparency, efficiency and trust).

This is not a “management summary” but a “summary for non-managers” (or non-decision makers), because I think that especially decision makers should understand more about a technology that is considered revolutionary, that they can ask the right questions to distinguish dazzle from real benefit.

The blockchain was created as a solution to a very specific, particularly complex problem: Creating a digital equivalent of paper money, trying to prevent counterfeiting while attempting(!) to preserve anonymity. For this problem, it may be a good solution when applied directly. For most other processes, it is massive over-engineering, as even small changes to the preconditions lead to much more efficient solutions. But to know and understand these factors, a deeper understanding is needed. And this is exactly the goal of this text.

In the course of its creation, the text has become larger than planned. However, it has also gained in readability and comprehensibility. In other words, it was written precisely with the target audience of executives and decision-makers (and grandfathers and granddaughters) in mind: Many memorable analogies and explanatory comparisons with the real world, also suitable to liven up a business or family dinner.

I think the text deserves a chance: the advantage of text is that you can skim individual passages at any time and ruefully read it later after all, without resenting it. The often exaggerated, colorfully highlighted introductory sentences to a chapter hopefully help with these reading decisions.

The blockchain, and with it the Bitcoin, emerged at the height of the financial crisis in 2008 as a countermovement to the role of the banks in this crisis, which first made profits with non-transparent financial products on bad mortgage risks and then, due to their systemic relevance (“too big to fail”), generated bailout money. Accordingly, the desire for transparency and independence from central organizations (banks, states) is a leitmotif running through all levels of the blockchain ecosystem.

This development bears the signatures of cryptoanarchy and anarcho-capitalism, i.e., of movements that want to avoid trust in organizations and structures and, in particular, distrust the state. Only programs (“code is law“) or free, i.e. unregulated, markets deserve trust. This trait runs through all levels, even though they are most visible in cryptocurrencies and smart contracts.

Promises

The only chapter blockchain enthusiasts would advise you to read.

Since blockchain and Bitcoin emerged at the same time, the term blockchain is colloquially sometimes also used (pars pro toto) for applications based on it (cryptocurrency, smart contracts, etc.) as well as the respective ecosystems. In the context of this series, I will try to clearly separate the terms.

The four to six properties that a blockchain has or should have are not uniformly defined:

WikipediaChristian Cachin/IBMMoreThanDigital
ChainingDistributed LedgerData integrity
Decentralized storageCryptographyReliability
ConsensusDistributed ConsensusFast storage
Manipulation securityBusiness processesAnalysis, transparency
Transparency,
confidentiality
Nonrepudiation

Let’s look at the four categories from Christian Cachin’s presentation as examples:

  1. A distributed ledger as a transaction list with an unchangeable past.
  2. Cryptography to ensure the integrity of accounting and authenticity of transactions.
  3. Distributed, failure-tolerant consensus on the content of the ledger and the validity of transactions: All blockchain participants should be able to agree on a common world view (e.g., account balances), even when not all participants are always available.
  4. Business processes (“business logic”), which are to ensure the validity of transactions and implement them (account management, smart contracts, …). We will defer this area to the next two articles, where they have the necessary context.

This is very abstract, especially as long as it remains unclear what is the actual goal to achieve. Since this article has set out to clearly structure the complex matter and we would like to be evaluated against this goal, we need to answer one more question first: What do heck do people want to use these features for? So let’s look at a typical “10 Applications for Blockchain” list and then extract or structure the benefits listed there:

  1. International financial transactions: Data consistency, tamper resistance, removing intermediaries→lower transaction costs and higher transaction speed.
  2. Healthcare: Storage in a distributed network allows storage of sensitive data: Patient records, medical findings, and clinical histories. Access only by users authorized by the owner.
  3. Identity management: ID documents can be digitally converted securely, with no data loss; more secure and faster.
  4. Prevention of money laundering: Transparency, record keeping, and attribution.
  5. Insurance: Smart contracts for claims processing, insurance fraud detection.
  6. Supply Chain Management: Simpler contracts, continuous tracking of goods, transparent documentation of supply chains from origin to store. Faster/simpler/cheaper investigations.
  7. Mobility: Access rights, documentation of ownership, automatic vehicle rental+billing, secure billing even with electric vehicles.
  8. Energy market: Dealing with change, traceability of transactions (private energy feed-in), billing/payment at e-fueling stations.
  9. Digital elections: No fear of manipulation, tracking every vote.
  10. Degree certificates: Issue degrees/certificates that cannot be falsified, internationally recognized.

These topics cover a wide spectrum, from blockchain itself to cryptocurrencies and smart contracts. It would be great if this could solve so many outstanding problems of modern civilization. However, since this hasn’t happened in the last 14 years, it doesn’t seem to be that simple after all.

­­­­­­­In the following (large) technology chapter, let’s take a look at the features and techniques used to solve these challenges before we move on to the reality check. Since we are focusing on the blockchain itself here, we will discuss the topics of data consistency/anti-tampering, data loss prevention, transparency/records and (partial) attribution/traceability/falsification at the end of this article We will look at the rest of the points in more detail in subsequent articles, after the necessary foundation stones have been laid.

Technology

The chapter in which the almost impenetrable jungle is explored and mapped.

Blockchain overview

Overview of the elements of the Bitcoin blockchain. On the left, the actual chain, a concatenation of blocks. Each block (center) contains, in particular, a list of transactions (right).

The technology of the lowest level alone, the blockchain, is quite complex. Before we go into depth, here is an overview, based on Bitcoin.

The Bitcoin blockchain consists of a chain of blocks (labeled “chain” in the right image) that grows over time. (In exceptional cases it can also branch, as we will see below.)

Currently, the Bitcoin blockchain is about 730,000 blocks long and a new one is added about every 10 minutes. Each of these blocks contains a collection of transactions. The whole thing is managed jointly (by consensus) by the computers (nodes) that build the system and where actually everyone can participate (no special authorization is needed).

To understand the interplay, let’s look at it in several small steps that together make up a table of contents for this technology chapter:

  1. The structure and function of the blockchain
  2. How to add a block even though no duty roster exists (the consensus)
  3. How nodes communicate with each other (the peer-to-peer network)
  4. Why anyone would do that to themselves (the incentive)
  5. How a valid block is generated (mining and proof of work)
  6. How else to settle this (Proof of Stake and friends)

This is followed by a short summary so that you can see the forest for the trees.

Before we analyze a digital blockchain, let’s build an analogy with paper and offices. We will watch anarchic accountants (or notaries) keep a tamper-proof loose-leaf collection. Those who want (and know the Hitchhiker’s Guide background) are welcome to imagine them as members of the Golgafrinchan “B” ark.

Analogy: notary’s office

The subchapter where we build ourselves an unchanging loose-leaf collection.

First stamped loose-leaf

Let’s start our approach to blockchain technology with an analogy from the analog world: an anarchic mix between accounting and the notary’s office. (As junior notaries, they are initially only allowed to juggle small financial assets. In later chapters, they are then allowed to notarize contracts and transfers of tangible assets. That is why we call them “notaries” even now, although they are more into bookkeeping for the time being).

For efficiency reasons, several notaries share the work. Because of the chronic back pain that came with it, the large, heavy notarial book with the collected confirmations of the last centuries has been abolished. Instead, a loose-leaf collection (much more popular with legal staff anyway) is used. So on the first day, the responsible notary fills out the list of transactions and confirms their accuracy with her official stamp.


Second loose-leaf with copy of the first

The next day, the notary on duty repeats the same procedure. But so that none of the notaries can later replace the sheet from the previous day with another, his list for today includes a scaled-down copy of the yellow list from the previous day. He stamps this compilation in the evening as usual to confirm its accuracy.

On the third day, the process is repeated with a third notary. This one also stamps his list in the evening together with a scaled-down copy of the list from the previous day, on which, transitively, again a scaled-down copy of the day before last can be seen and so on.


Third loose-leaf with copy of the second, on which the first is also visible

Continuing this process, the notaries generate a common chain of documents in a coordinated manner. In this chain, no previous document can be exchanged without simultaneously adjusting all subsequent documents as well. And that would definitely attract attention.

But coordination of the stamping right and duty also requires trust and recognition of authority. Below we will see how we can solve the paper blockchain without this prior agreement (and thus trust in anyone), because this is also the cryptoanarchic basis for the digital blockchain.

For simplicity’s sake, our junior notaries have a set work schedule for now and enjoy their time off, so there are no conflicts over who is responsible for which day. This will change as we get closer to understanding the real blockchain.

Digital Blockchain

The subchapter where we digitize the loose-leaf collection with Lego bricks.

The blockchain as stacked Lego blocks

Let’s switch to the digital blockchain, such as the one used by Bitcoin. It works very similar to the loose-leaf collection above: Each block consists of a list of transactions and someone asserts the veracity of the list. This someone is generally called the “creator” here; in cryptocurrencies, they are commonly know as “miners”.

A cryptographic checksum, a so-called hash, can be calculated for each block, and the checksum of the predecessor block is then stored in the current block, analogously to the scaled-down copy of the previous day in the notary example above. This means that the new block is irrevocably bound to its predecessor block and, indirectly (or transitively), to all of its ancestors.

Each block is unique (especially its end plate)

For a more detailed understanding, let’s tune our stackable blocks a bit: Instead of always having the same well-known regular Lego pattern, where each block matches every other block, each block now has a unique dot pattern on top, corresponding to its unique hash. The following block must now-like a matching lock and key-have the exact matching hole pattern so that the two blocks can be stacked.

The next block must have the matching holes, otherwise stacking will fail

The properties of the hashes used are such that even if all the computers in the world did nothing else for billions of years, they could not find a second block that had the same hash value. The end result is that we have a perfect reduction of the previous block stored in the current block: Instead of the 1-2 megabytes that a typical predecessor block occupies, only 32 bytes, about 50,000x less, are used.

(In contrast to the paper analogy, where it is still possible to read out all details directly from the slightly reduced copy, it is impossible to read out any contents of the predecessor block from the hash alone. The actual purpose of both paper copy and hash-the unique identification of the predecessor block-is, however, perfectly satisfied in both cases).

Wanted: Notaries without duty roster

The subchapter where we liberate the notaries and finally allow them to live and work independently.

Our notaries used to have a clear duty roster: On each day, exactly one person was responsible for keeping the list. The list was created by mutual agreement or by assignment of their boss. Bitcoin is a brainchild of both crypto-anarchy and the outrage at the behavior of the central power structures in the financial crisis; accordingly, there must be no supervisor role of any kind there! No matter how useful a boss is in increasing the efficiency of coordination and creating rules, at some point a boss could abuse their power and there would be no mechanism within the system to prevent it.

In the real world, there are mechanisms for deposing abusive or corrupt superiors, but this dismissal can sometimes be lengthy and laborious. Real-world mechanisms to prevent and detect bad developments at an early stage include honesty, empathy, and trust. These evolutionary advantages lead to avoiding such situations or remedying them quickly. However, these require regular interaction and therefore do not scale to 8 billion people. And they do not fit into the crypto-anarchic world view anyway, according to which one does not want to make oneself dependent on anyone.


Probably the best-known examples of self-organization
are provided by schools of fish and flocks of birds. Maybe some of you even remember the exaggerated (and thus particularly catchy) depiction of the moonfish in “Finding Nemo”.
(Image: adiprayogo liemena at Pexels)

This leaves only agreement between the notaries as a way out, the above-mentioned consensus. However, not only do we not have a boss who makes a duty roster, we also do not have a boss who hires notaries or calls them back if one of the notaries prefers to go fishing on their scheduled work day.

That means, we need a system that has to cope with a constantly changing number of notaries, all of whom do not want to take an aptitude test beforehand and do not want to announce their presence or absence beforehand. The purest chaos. Or, somewhat nicer, a self-organizing (or, even more educated, autopoietic) system.

Human groups are much less homogeneous than schools of fish; accordingly, there are hardly any self-organizing groups with more than a dozen people. For larger groups, sooner or later structures for the division of labor therefore emerge. In standardized work processes, these structures reduce the coordination effort and thus increase efficiency. But it is precisely these structures with the associated power positions and dependencies that one wants to avoid with cryptocurrencies. This is at the expense of efficiency, as we will see next.

Organizational structure

The subchapter in which notary publics learn to communicate with each other from their home offices.

A network to bind them all

For our paper-based notaries to form a notary’s office, we need to have both

  1. an organizational structure (in this subchapter) and
  2. an incentive or punishment system (carrot and stick), i.e., a substitute for bosses and wages, which we look at in the upcoming chapter.

First, the structure: All those who want to try their best at being notaries become part of a network of equal partners, also known as a peers, for the purpose of exchanging information. In such a peer-to-peer network, (1) transactions which are to be included in the blockchain are communicated to everyone, and (2) the newly created blocks (or, in the analogy, the stamped paper slips) are stored by everyone.

This peer-to-peer system is a set of computers (“nodes”, in the figure on the right the circles with the letters) and a number of connections (channels or “edges”) between them, over which messages can be exchanged. Now, if the blue node A has a new message, it sends it to all of its neighbors, as indicated by the blue arrows; in this case, nodes B and C. If the information is new for these two and passes their validity checks, they in turn send the information on to all their other neighbors: Thus B sends the message to C and D (red arrows). A is left out, since B has received the message from there and thus knows that A already must have this information.

C in turn sends the message to its neighbors B, E, and F. On the connection between B and C, the message travels once in either direction, since at the time of sending, B and C have no way of knowing that their respective peers already know the message.

This so-called “flooding” creates a (tiny) tidal wave, sweeping through the entire system of channels, reaching all the nodes from A to G. Many of the participants receive the message multiple times, in the worst case once from each neighbor. Not very efficient, but that’s how it is when you can’t or won’t trust anyone.

Finally, this means for our notaries that they can write off any thoughts of ever having any free time again. While this is not particularly conducive to motivation, it is robust against failure (or even malice) of individual participants. Lacking central authority, there are no options for sanctions, so everyone must look out for themselves first and rely as little as possible on anyone else. From a cryptoanarchic point of view, this waste is better than trust; a pattern that accompanies us again and again throughout the entire ecosystem.

Carrot and stick

The subchapter where we motivate freelancers.

However, the above-mentioned flooding of messages through the notary network is only a tiny part of the daily grind: our notaries not only have to constantly forward all messages, but everyone also has to constantly update the transaction list, because you do not trust your colleagues to do it right. On the other hand, this also means that from now on you have no free time, because you are busy working around the clock and checking the work of your colleagues. And even this is done in a very inefficient way.

But first back to real-world day-to-day work: One way to keep a team “on track” is to create an incentive system with defined criteria and rewards. With human teams, verifying that criteria have been met is very difficult, even if it has been attempted with piecework, for example; this is know to reduce (good) intrinsic motivation to (not as powerful) mostly extrinsic motivation, known as the Overjustification effect. This is hardly sustainable, as quantity comes at the expense of quality, or team members try to make their own performance look better or belittle the performance of colleagues: Intrigue, concealment, lies, and manipulation, possibly up to sabotage. These are key elements to guarantee suspense in movies and novels, but in reality they are highly detrimental to productivity and the achievement of common goals.

In our exotic notary’s office, processes are simple, repetitive, and standardized, which is why most blockchains assume objective measurability of work performance. Thus, a monetary incentive system was created for Bitcoin, which awards a reward (in Bitcoin) based on concrete, verifiable rules for the creation of a block if this block is generally recognized. This is why this procedure is called “Proof of Work” (PoW).

So far, so simple. But how can you punish someone if they have not done their work well enough? You make the work so strenuous and tedious that no one does it voluntarily unless they can expect to be generously rewarded for it!

Thus, the incentive is based on one of the strongest motivators humans have: Greed. Greed for money and power. And this incentive also works perfectly, at least superficially. But precisely because of its now corrupting strength, greed leads to the inexhaustible human inventiveness being put to its service and loopholes being sought and found. More on this after the facts in the “reality” chapters in this and the next post.

Work and wages

The subchapter in which honest cooperation is finally rewarded.

Anyone who wants to add a new block to the Bitcoin blockchain receives a reward of currently 6¼ Bitcoin (₿) for creating (“mining”) the new block, which is currently equivalent to just under a quarter of a million Swiss Francs (or, approximately, $ or € or £). Not bad for 10 minutes of work! And no wonder greed has become the driving force. (More on finance and its role in the ecosystem in the next article).

If you want to claim that quarter of a million, what do you have to do?

  1. Collect incoming valid transactions (more on this then in the next article on cryptocurrencies and transactions).
  2. Include as many pending transactions as possible in the new block to be created.
  3. Process the block until “enough work” has been done (explained next).
  4. The creator (“miner”) of these blocks then receives the 6¼ ₿ reward as well as the transaction fees of all transactions included in this block (the collected transaction fees of a block only amount to roughly 1 % of the mining reward, as of today). Let’s hope this is you!

This so-called d100 (or “Percentile dice”)
has “only” 100 faces. On average, it shows a 1 every hundredth roll.

“Sufficient work” is done when a cryptographic puzzle has been solved that requires repeated, arduous trial and error. For example, any mining computer on the Bitcoin network could tirelessly roll a 120-sextillion-sided die (1 sextillion is a 1 followed by 21 zeros!) until it scores a 1. This successful dice roll is obtained on average after 120 sextillion attempts. With this result, the “Proof of Work”, the miner can claim the block and collect the reward.


Our notaries incessantly drop 120 sextillion stamps from the ISS

­­Our notaries do not play dice, of course. But they love hide-and-seek games, are very forgetful, and at the same time passionate space tourists. That’s why, when their transaction collection work is done, they hide their completed transaction list somewhere on the 510 million square kilometers of the Earth’s surface, immediately forgetting where they put it, and head off into space. From there, they throw their rubber stamps as fast as they can onto the earth’s surface until one hits the 6 cm by 7 cm stamp field of their transaction list. (510 million km² / 120 sextillion ≈ 42 cm². 42. honestly! Hopefully nobody seriously gets the idea to use the earth as a giant ball…).

Computers do not roll dice or throw stamps, since cheating would be too easy with both techniques. Therefore, when attempting to “mine” a new block, a specially designated field in this new candidate block is filled with a random content until the value of the checksum (“hash”) over the block is sufficiently small; as small as it gets only once every 120 sextillion times. This sufficiently small hash, after countless trials, is thus the proof of our work. (Incidentally, the hash of the block cannot be predicted without actually computing it, i.e., there is no shortcut).

The more computing power there is for mining, the faster this puzzle is solved. However, a design decision of the Bitcoin protocol states that a new block should be added every 10 minutes on average. To ensure that this remains the case as computing power increases or fluctuates, the definition of “enough work” must be regularly adjusted to reflect the computing power available on the network. In Bitcoin, this adjustment of the difficulty happens every two weeks.

In the dice analogy, this would mean that if there are more or faster participants in the network, the dice would also be rolled with a correspondingly larger die, and thus the dice would have to be rolled more frequently until a one appears. (In the case of our space notaries, the size of the target field for the stamps would be reduced).

This results in the following:

  1. It depends extremely on chance who “mines” the block and “wins” the quarter of a million. To achieve more even, predictable payouts, miners gather in pools, which then divide the proceeds among themselves.
  2. It can happen that several nodes get lucky almost simultaneously and find one of the countless (around 1 septendecillion, a 1 with 54 zeros) possible solutions to the cryptographic puzzle. Thus, several valid new blocks exist simultaneously. In general, these blocks differ by the list of transactions included in them, so they are not interchangeable. Accordingly, the system must make a choice as to which block to continue working with. Details are given below.
    All others who have unsuccessfully “rolled the dice” have another chance (like in the lottery) with the next block. At some point the big win must come…
  3. While the carrot is the chance to win (like a played lottery ticket), the stick could be seen as the wanton tearing of the lottery ticket: If you have made the immense effort to mine a valid block (“rolled a one”), but the other nodes in the Bitcoin network don’t accept that block because you made a mistake (e.g., recorded an invalid transaction), then you forfeit your chance to win and are guaranteed to remain on the accrued costs (electricity costs for computer and cooling, rent, amortization, …) without even a hint of a chance to win.
  4. As long as someone believes that his or her accumulated costs are lower than the long-term expected share of a block’s proceeds, they will try to set up additional mining capacity. I.e., computing power follows the price of bitcoin as long as no bans change the rules).
  5. Whoever can get electricity (and cooling) cheaply is at an advantage. As a result, mining tends to be done at locations with low electricity costs, which is often coal-fired power. (Unless you “get” the electricity for free).

Collapse of mining rate in China (yellow) after de facto ban on cryptocurrencies (Image: CBECI).

The Cambridge Bitcoin Energy Consumption Index (CBECI) assumes an annual electricity consumption of 125 TWh for bitcoin mining, the Digiconomist of 200 TWh. At the (for western Europe: unrealistically) low electricity price of 5 US cents per kWh assumed by CBECI, electricity alone would cost 6.25 or 10 billion USD, or 20 or 32 million dollars per day(!), and the trend is rising.

This high energy consumption is also brought up in between as an argument for banning proof-of-work cryptocurrencies. As high as this effort is, it should (like all political decisions, but not only these) always be seen as the result of a cost/benefit analysis. For this, I defer to the subsequent cryptocurrency article.

Immutability in the blockchain

The subchapter in which we try our hand at forgery and fail gloriously.


Finding a matching replacement block is “impossible”

One of the important arguments in favor of a blockchain is its immutability. Thus, replacing an existing, established block in the blockchain with another at a later date is not possible without also laboriously adapting each of the subsequent blocks (remember the loose-leaf analogy). That is, we cannot simply take one of the blue-purple blocks (for example, the one labeled “OK” in the image to the right) and replace it “just like that” with the cyan block (“BAD”), which is why it is also crossed out in red. This is due to the security of the cryptographic hash function used.

What would have to happen to replace the block “just like that”?

The Bitcoin network combines a massive computing power for its proof of work, and consumes 500 times more electricity than the world’s most powerful supercomputer. And despite this phenomenal computing power, the entire Bitcoin network would have to calculate for 20 octillion years to replace one block with another so that its checksum (hash) would be the same as that of the original block, i.e. it could be accepted as a valid member of the chain. (Whether it would actually be accepted is another matter).

20 quindecillion years are written as 2 with 49 zeros. Who prefers comparisons to the age of the universe: all those computers would have had to calculate continuously since the big bang more than 13 billion years ago and still would only have completed a minuscule fraction (one dodecillionth; a dodecillion being a 1 followed 39 zeros!) of their Sisyphus work. (And even Deep Thought would probably find the answer to the “question of all questions about life, the universe and all the rest” faster than a replacement block).

People from the IT security environment cautiously call this “impracticable”; ordinary mortals, however, prefer “impossible”.

Alternative rules

The subchapter where notaries look for a less repetitive, easier job.

The proof-of-work rules inevitably lead to energy waste, which usually only increases with time or popularity. This pleases neither environmentalists nor the operators of the mining computers (“rigs”) themselves: They would rather rake in more money themselves than spend it on electricity and hardware. Accordingly, alternative systems have been and are being considered, in particular Proof of Stake (PoS) and Proof of History (PoH).

The goal of all these rules is to achieve security, speed, and decentralization at the same time. Unfortunately, no one has yet managed to achieve all three simultaneously in an open blockchain. Despite intensive attempts, trade-offs have always been necessary so far, a realization which is referred to as the “blockchain trilemma“.

Proof of Stake

None of the prominent blockchains uses proof of stake productively as of now. Ethereum, the number two behind Bitcoin and based in Zug, is currently still working on a proof-of-work basis, for which more than half of Bitcoin’s power consumption is currently used. Ethereum has wanted to introduce PoS productively for years; after several postponements, it is currently scheduled for the second quarter of 2022, parallel to PoW, which is still running.

What is Proof of Stake? The underlying assumption is that the more shares of a cryptocurrency you own, the more likely you are to care about the welfare of the cryptocurrency as such and thus would not act against the interests of the cryptocurrency. (Since this would assume an extremely homogeneous humanity, this claim certainly does not cover everyone.)

Let’s start again with an analogy, a self-organizing village. The villagers feel disturbed in their work and leisure time day and night by the hectic running around of the proof-of-work notaries. Due to their lack of sleep, the notaries have also become very anti-social; no one wants to have anything to do with them anymore. Therefore, the Council of Elders decides to switch to Proof of Stake, and thus from (proof-of-work) miners to (proof-of-stake) validators.

Here is the PoS process using the example of the validator committee responsible for March:


Timeline of committee activity for Proof of Stake (PoS)

The stupid, repetitive work of PoW has been dropped at the expense of much more complex coordination. At the level of abstraction shown above, it may sounds quit simple and convincing. However, the above description is massively simplified compared to the actual function of PoS on Ethereum.

Additional complexity not covered above comes from the fact that Ethereum splits this work across 65 blockchains (one main or coordination chain, the “beacon chain”; and 64 transaction chains, the “shard chains”)

However, the greatest complexity in any distributed system always arises from the detection of errors and the handling of special cases. Countless rare corner cases have to be handled sensibly and it has to be ensured that everything continues to run, even if something goes wrong: Whether it is a proposer or other validator calling in sick, being overloaded, or otherwise unavailable; when there is disagreement between proposer and other validators; when someone falls asleep during the elections; and many more. In the case of Ethereum, all are implemented in an automated fashion with appropriate penalties for the wrongdoers and rewards for the rest. Even under the most unfortunate combinations of errors, the system must always be able to continue to operate in a meaningful way.

Random numbers are a special weak spot in this game. They are needed e.g. for the random, fair selection of committee members or the distribution of proposer activities among the members. In a blockchain environment, however, randomness is anything but simple, and the seemingly trivial task is implemented in Ethereum-PoS with an elaborate “smart contract” with (1) stakeholders of their own, (2) even more cryptocurrency put at stake, and (3) carrots and sticks again with credits being distributed and collected. This random number smart contract, dubbed “RANDAO“, is one of the many tiny cogs in the whole process. (A separate future article is devoted to how smart contracts work.)

As we can see, the blockchain itself can only function if countless smart contracts constantly interact tightly and correctly, like a finely tuned clockwork. In turn, however, the proper functioning of the blockchain also depends on these smart contracts. Computer scientists try to avoid such cyclical dependencies in distributed systems at all costs, since they lead to more complexity and additional single points of failure.

Full membership in the Ethereum-PoS VIP lounge starts at 32 Ether, currently available for around 80,000 francs. Parts of this stake or even their entirety can get lost if a computer crashes at the wrong moment or its network connection is disrupted (e.g. during the “RANDAO” mentioned above).

Risks with Proof of Stake vs. Proof of Work

If a proof-of-work machine fails (crash, power failure, network problem, DoS attack, …), its chance of making a profit may be reduced, but any capital earned before remains untouched.

With Proof of Stake, on the other hand, you want to “stake” as much of your assets as possible, since the profits are distributed in proportion to the capital staked. And if the computer fails at a bad point int time, your entire staked capital is gone.

Permissioned Blockchains

In addition to public, open (“permissionless”) blockchains based on the Bitcoin model, so-called private or “permissioned” blockchains are popular for closed user groups in business applications: To join the closed user group, traditional contracts with paper and signature are concluded. The members are then each allowed to integrate one computing node into this closed system.

These computer nodes then ensure that new entries are only added to the blockchain if they receive at least a ⅔-majority of approval. The system is thus immune to Byzantine errors, i.e. the system functions correctly as long as less than ⅓ of the nodes fail or act erroneously/maliciously.

In such a controlled environment, detecting or preventing dominant players is much easier than in a model where anyone can just join or leave at any time, no questions asked. In contrast, the permissioned model also lacks the universality that a public blockchain strives for.

Further, the permissioned model is mostly used for applications other than currencies, so it is typically not self-funded. The “miners” cannot be rewarded with something inherent in the system (e.g. a Bitcoin). Instead, somebody has to pay for the cost of the system (computers, electricity). That “somebody” is typically an organization formed specifically to operate the blockchain. But then the scheme is no longer fully decentralized: a centralized authority is funding it.

At the top it wobbles the most

The subchapter where we resolve Lego tower building conflicts and recheck stability.


Sometimes, there be more than one (simultaneous) new block

Since 2009, the Bitcoin blockchain has grown to a height of now over 730,000 “Lego blocks”. We have seen that it is “impossible” to pull out a block at the bottom and replace it. After all, it is Lego and not Jenga!

But what does it look like at the top? In the mining process, which is characterized by luck (rubber stamp or dice throwing), several new valid but different blocks can be created practically simultaneously. It often takes several dozen seconds, sometimes more than a minute, for all nodes to be informed about a new block. About every 11 days, therefore, two potential newest blocks are in wide circulation at the same time. But there can only be one, because consensus on the account balance must somehow be guaranteed.

Therefore, the top few blocks are considered unsafe, often until about six blocks have been stacked above them, which takes an hour on average, but can take up to 6 hours (the waiting time only depends on dice throwing luck). Then the probability is high that an additional, longer chain will not suddenly appear somewhere.

This leads to further conclusions:

  1. If there are multiple candidate blocks in play, each participant chooses one of the candidates as the basis for the subsequent block, regardless of the decisions of all other participants. (Typically, this is the first valid block he has seen).
  2. Candidate blocks that the majority is not “happy” with remain in irrelevant side arms of the blockchain and are therefore ignored.
  3. The “happiness” is defined by the rules in the program code and may change over time.
  4. In other words, the majority determines the course, even if there are individual stubborn minorities who then ride their special train (“hard fork,” a kind of declaration of independence). This mechanism has also been used for targeted discrimination or exclusion of members and technologies.
  5. A transaction that only appears in one of these sidearms will not be recognized by most members. But if the transaction is basically valid, it should be included in one of the next regular blocks and thus eventually come to recognition.

Technical summary

The subchapter which tufts everything again nicely.

Millions of computers with a power consumption of 500 supercomputers or an industrialized country are constantly trying to solve a cryptographic puzzle. The puzzle difficult increases as more or faster computers take part in it, so that on average a new block is created every 10 minutes. Solving this cryptographic puzzle (“proof of work”) is intended to confirm that one is sufficiently interested to seriously commit to the principles of the blockchain, i.e. not to cheat.

However, if enough people do not agree with the rules (or cheat together), these rules become the de facto standard (“Code is Law“). This is how changes are implemented or spin-offs or exclusions occur.

The incentive to provide these computing resources, to bear the enormous power costs of 20-30 million francs a day, and to abide by the rules is based on the freshly minted cryptocurrency (currently around a quarter of a million francs every 10 minutes) created from nothing and the transaction fees, currently a few thousand francs per block. In turn, as we will also see later, there is also a desire to make the rules so that the income or value of the cryptocurrency becomes as high as possible.

Alternatives

The chapter where we review other Creations.

From the perspective of blockchain aficionados, the world was desolate and empty before Satoshi Nakamoto separated the world into light (blockchain and Bitcoin) and dark (everything else) on the first day and then retired; perhaps to watch the price development of his first million Bitcoins with relish.

But digital archaeologists, working tirelessly, have discovered that important approaches existed even before that:

Prehistory and early history

The subchapter where we travel back in time before the Bitcoin Big Bang.

Image source: Alan T. Sherman, Farid Javani, Haibin Zhang, and Enis Golaszewski: On the Origins and Variations of Blockchain Technologies, IEEE Security & Privacy, Jan/Feb 2019

David Chaum, the busy cryptographer we will meet a few more times in this series, already dedicated himself to a blockchain-like protocol in his 1982 dissertation “Computer Systems Established, Maintained, and Trusted by Mutually Suspicious Groups”.

In 1990/91, Stuart Haber and W. Scott Stornetta addressed the issue of the traceability of digital processes by means of time stamps, digital signatures that confirm the existence of a digital file at a certain point in time. In particular, they investigated the question of how to make the issuance of falsified (backdated) time stamps transparent and thus recognizable as forgeries. A variant of their method has been in continuous use since 1995 as the PGP Digital Timestamping Service.

Timestamps are especially useful when you want to prove the existence of a document at a certain point in time or to show that a document has not been modified since this point in time. Application areas are diverse and range from the patenting of inventions to copyright and use in forensics.

In 1997, Ross Anderson described the Eternity Service, an anonymous, decentralized storage medium that protected documents against denial of service. Since 2000, the LOCKSS (Lots of Copies Keep Stuff Safe) system has been protecting digital documents from loss or corruption in a decentralized manner. LOCKSS research also resulted in the first (award-winning) proof-of-work system for consensus building in 2003.

Since 2001, there have been distributed systems for tracking the development of program source code, in which the current state references the previous state by referencing its hash (the “checksum” mentioned repeatedly above), thus making backward substitutions of previous states impossible. The source code management system Git, published in 2005 and written for the distributed development of the Linux kernel, has become the dominant form of source code management for open source projects and is also indispensable for enterprise software development. As such, Git could arguably be described as the world’s most widely used blockchain solution. In contrast to the previously described systems, each developer can grow their private chain as they wish. Consensus is reached through (human) dialog. Multiple chains are not really a problem and when a new consensus is established, the work done in a distributed manner is not lost.

Blockchain Trilemma

The subchapter in which we learn the hard way that we get everything at the same time.

Not all three at once

The blockchain trilemma postulates that you can achieve a maximum of two of the three following implementation attributes simultaneously: security, speed and decentralization. There is no formal proof of this; however, until now, every blockchain project has run up against this barrier in some form or other.

Accordingly, any assertion to the contrary should be taken with extreme caution. Remember: “If something seems too good to be true, it probably is.”

Can it be a little less?

The subchapter where we see once again that if we want less, we get more.


Everything becomes easier if you can omit one of them (see enumeration)

From the point of view of functionality, we can also form a triangle:

  1. Reaching a global consensus (everyone sees the same state in their books, the distributed ledger),
  2. the complete absence of any trust between the participants; and
  3. the desire for everyone to be able to enter their own data if they only meet certain minimum requirements; they do not have to go to a coordinator to do this.

Achieving all three goals simultaneously is very costly, as we have seen. If we could omit at least one of them, the problem would be much easier to solve. Let’s therefore look at them one by one:

Do we need global consensus?

Let’s divide the consensus problem into three cases: Private blockchains among contractors, public blockchains for traceability only, and public blockchains with cryptocurrencies.

Are the expensive extras really necessary?

The main use of private blockchains is for business partners to assure each other of certain things. According to the nature of things, these are mostly bilateral contracts. No global consensus is required for this, only the two business partners have to agree. This is much simpler and can be implemented much more efficiently than a global consensus. The complexity of the blockchain is therefore completely unnecessary or even harmful (slower response times, unnecessary publicity, costs, …).

In the case of a public blockchain on which a currency or other elements from the subsequent articles are built, global consensus seems important at first glance, since everyone should be informed about every account balance. But even this is not mandatory, as we will see with alternative payment solutions; local consensus or consistency (which is much simpler than consensus, as it only requires the absence of contradictions, instead of perfect agreement) is often enough. But more on this in the article on cryptocurrencies.

If public blockchains are only used as information repositories, as is the case with most blockchain-based public sector digitization projects, global consensus is entirely unnecessary. Integrity, transparency, and proof that the information has not been tampered with are quite sufficient, and these attributes are much easier to achieve. If I want to show that a certificate has not been altered, a digital signature from a trusted authority is sufficient. Although immutability is regularly cited as a goal, it often is not actually desirable. Errors such as typos or vandalism should be able to be corrected, and the right to be forgotten also has to be taken into account. That is, entries should be able to be corrected or deleted, but these changes should be traceable, either only by supervisory bodies or by the general public, depending on the actual application.

(The technically most expensive and most complicated of the additional options is the protection against using a single piece of currency multiple times in parallel, aka “double spending”. We only need it if the blockchain is to keep records of things that, like money, must only be spent once. More about this in the upcoming article on cryptocurrencies).

Do we need write permissions for everyone?

In many use cases, there are only a few authorized writers; ideally, they even belong to the same organization (commercial register, land registry, …). Alternatively, they are known to be responsible for only part of the data (hierarchy, federalism, …). I.e. even if consensus is necessary, it is clear who may contribute what to this consensus. Again, this leads to much simpler solutions.

Would you like a sprinkle of trust with this?

The huge effort that goes into proof-of-whatever and fully automated consensus is eliminated if there is at least a modicum of trust and external mechanisms (press, courts, …) can be used if this trust is abused.

Reality

The chapter where we revisit the original promises.

Let’s look again at the promises for which blockchain is being sold as a panacea.

A remark in advance: As the topics in the promises are only stated in very general terms, no detailed analysis can be made. Nevertheless, I have tried to make this more specific by adding my personal expectations from such a system as requirements.

The topics were analyzed according to eight criteria:

The purely financial topics “financial transactions” and “money laundering” can be found in this table; nevertheless, they will be dealt with in the next article.

TopicInte­grityTraceabilityImmutability>2 OrgsConsensusDouble SpendingLayer 8Idea
Financial transactions(✅)(✅)(✅)
Health care😨
Identity management
Money laundering
Insurances(✅)
Supply Chain(✅)
Mobility
Energy markets
Digital elections(❌)😨
Digital certificates(✅)

Let’s take a look at a few hot topics as examples:

Digital elections

The subchapter where we shudder again more about the idea of digitizing elections.

e-Voting was a hotly debated topic in Switzerland a few years ago, so here are just a few key points. Trust is essential when it comes to voting:

Therefore, we want integrity and immutability, but traceability on a large scale is already dangerous.

We do have a strict organization with hierarchy (federal, cantonal, municipal level) and not a horde of mutually distrusting individuals; thus, global consensus is ensured simply by passing the results to the next higher level of hierarchy (and verifying that they are included in the result).

Electronic systems of various kinds have turned out to be unreliably, easy to manipulate or faulty or insecure. Even in the absence of these technical, but in part also fundamental, issues, the problem of intransparency or lack of traceability remains: Currently, the paper-based vote counting process can be observed in many Swiss municipalities. Thanks to this critical public, some problems with e-counting in the city of Bern could also be identified and have led to improvements.

This means that electronic voting may seem sensible, but it carries high risks of electoral fraud or (even worse) a loss of trust, and it offers hardly any time advantage (at least in Switzerland, the results are usually known a few hours after the polls close). Despite the intense discussion, no one has yet even begun to show how a blockchain could help there (beyond hand-waving general claims). On the contrary, I could well imagine that such systems would be the wet dream of any totalitarian regime.

Accordingly, I see no reason to think electronic blockchain voting is anything but a horrible idea.

Degree Certificates

The subchapter where only failing grades are given.

In Germany, a project to blockchainize school report cards has just been scrapped without results after spending over €1 million.

First, there are fundamental concerns that any guaranteed genuine digital documents would increase pressure for data or identity theft. On the other hand, it is questionable how the quality of the data will be guaranteed, along the lines of:

“The blockchain can’t lie to you, but you can lie to the blockchain.”

(This is, by the way, half of the Blockchain Oracle Problem.)

Third, this also raises the question of what blockchains would contribute to Degree Certificates:

Furthermore, essential basics of secure design of (not only web) applications were violated when implementing the application, throwing generally accepted design criteria such as use of few, simple tools, don’t trust any user input, … in the wind. These things should be considered and built in from the beginning, not only if you want to build a particularly trustworthy application.

This project not only shows that fundamental questions seem never to have been asked. Its use of 16 (!) blockchains raises big questions. Already a year ago, an expensive prototype of a vaccination certificate (which was not put into operation afterwards) was publicly criticized by various sides without understanding the pompous use of 5 blockchains. The new increase to 16 blockchains not only looks weird, but IMHO only allows (either of) two conclusions to be drawn:

  1. None of the 16 blockchains is reliably expected to continue to function and be secure in the future, i.e., to be able to play its primary role as a persistent, trustworthy storage.
  2. Too many special interests were represented in the consortium: Everyone wanted to push “their” blockchain.

Neither speaks to confidence in blockchains and their proponents.

Both projects’ (proof of vaccination and certificates) use of multiple blockchains also points out that consensus is irrelevant. Instead, everything that matters is the (much, much simpler) proof of creation before a certain point in time, a.k.a. timestamp.

(Incidentally, this project also provides another frightening example of how technical implementations often completely ignore the needs of an important user group. The students, for example, who should certainly be important beneficiaries of this technology, were expecting the project to culminate in something entirely different: timely feedback if their performance is unsatisfactory and what they can do about it. This has nothing whatsoever to do with digital signatures on diplomas.)

Supply chain 1: Provenance/origin

The subchapter in which we repeat ourselves again. On purpose.

Were the bananas really grown under humane conditions? Are the medicines on my table genuine or are they not potentially dangerous counterfeits? These are the kinds of questions that blockchain technology is supposed to solve. Let’s keep it short, as many ideas have already been explained with certificates, above:

In both cases, the blockchain is not the solution, only trust (and control) can help. Yes, this requires human contact and the odd on-site visit; but anything else is nothing more than an open invitation to fraud. Deal with it.

Supply Chain 2: Contracting

The subchapter where only standard processes help.

The second promise around supply chains is easier and faster formation of contracts. A contract is only really important when the contracting parties disagree. Then it must be watertight, with clear rules on when the contractual obligations are deemed to have been provided and what dispute resolution mechanisms are accepted. As long as all is sunshine and roses, a rough definition of the respective duties is more than sufficient.

A contract in a blockchain (and even more so, as we will see, a smart contract) must also be suitable for those bad conditions. A real speed advantage only arises if one can rely on the cooperation of the counterparty (common interest, trust, small risk, …) or can reuse elements from standardized contracts.

Evaluation

The chapter in which we look back disillusioned.

Here is a brief summary of the preceding pages:


Misapplied security mechanisms, including blockchain, are a daunting obstacle for the good guys, but no protection against the bad guys. (Image source unknown)
Possible decision diagram for Blockchain decisions according to DHS S&TD. (Image source: Recreation of NIST IR 8202: Blockchain Technology Overview, 2018; p. 42, PDF page 53)

From this we can draw the following conclusions, among others:

For every possible digitization approach, dozens of proposals exist for the use of blockchains. However, despite millions of dollars of investment in more than a decade, there is a lack of widely publicized recipes for success that leverage the specific properties of blockchains. Several aspects and considerations behind the concept of blockchain are inspiring and promising, but these inspirations have not yet been successfully implemented. Perhaps this is because blockchain really does not add value outside of the cryptocurrency environment; or perhaps it is because there has been a lack of understanding of how it works. I hope to have contributed something to this understanding and would be happy to hear about successful (but also unsuccessful) digitization projects.

Questions

The chapter to go.

Here are some questions that help to structure the digitization of a data-centric business process. If this process is to be specifically digitized by means of blockchain, consensus on the answers between all parties involved (“stakeholders“) is a must before starting the discussion “Blockchain? If yes, how?” should be started. I hope these questions support as many projects as possible in dealing with their data more clearly!

— — — — ✀ — — — — — — — — — — — — — — — —

Data lifecycle questionnaire

  • Data: Which data with which properties (structure, dependencies, data protection) are we talking about exactly?
  • Structure: What are the format, structure and dependencies of the data?
  • Criteria: Are integrity, traceability, immutability, confidentiality, or global consensus necessary?
  • Input: What are the data sources? How does the data get into the system?
  • Quality: How is the quality (correctness, uniformity, authenticity, completeness, …) of the incoming data ensured?
  • Error culture: What can/should/must happen if erroneous data (intentionally or unintentionally) creeps in?
  • Format changes: How should future changes to the structure or format of data be handled?
  • Data protection: How should the right to correction or deletion of personal data be implemented?
  • Processing: What processing steps should be performed on this data?
  • Trust: Is lasting mistrust between the actors to be expected? Can this mistrust be managed through hierarchies or (work) contracts?
  • Output: What should happen with the data?
  • Effects: What actions should happen (automatically) based on these results?
  • End: Do the data have an expiration date? What should happen then?
  • Universality: Do these properties apply equally to all data?

— — — — ✀ — — — — — — — — — — — — — — — —

As soon as data has to be changed or deleted, this data is probably not suitable for the blockchain. But there are alternatives.

This questionnaire on the lifecycle of data, not only for blockchains, is also available in a layouted, organized and expanded form.

Further reading

Let’s stay in touch!

Receive a mail whenever I publish a new post.

About 1-2 Mails per month, no Spam.

Follow me on the Fediverse

Web apps


Leave a Reply

Only people in my network can comment.

This site uses Akismet to reduce spam. Learn how your comment data is processed.