Monad consensus client is now open source (link below). This is the result of thousands of hours of effort by the team at @category_xyz. Enjoy

Summarizing the breakthrough in MonadBFT Yesterday Category Labs released the MonadBFT paper, describing the consensus mechanism that will power Monad at mainnet. MonadBFT is a significant development in consensus research since it is the first time that Pipelined HotStuff becomes resistant to tail-forking. Tail-forking occurs when a missed slot causes the previous proposal to be discarded and re-mined. It is a severe problem in previous Pipelined HotStuff formulations since it opens up multi-block MEV attacks that destabilize consensus. Alleviating this problem is a huge deal because it gives us all the benefits of Pipelined HotStuff - frequent blocks, low latency, large validator sets - while avoiding the biggest downside. MonadBFT also offers a huge upgrade for finality. It features single-slot (500 ms) speculative finality and two-slot (1s) hard finality. “Speculative finality” means “finality that will revert only in the event of equivocation (double-signing) by a majority of validators”. Equivocation is a major offense in most blockchain systems and is commonly penalized with slashing; the bigger the penalty for equivocation the closer you can think of “speculative finality” to finality. One-slot speculative finality is a huge unlock for high-performance applications, which can confidently display the updated state of the world immediately after the next block is received. These properties make MonadBFT a huge advancement in consensus, and a worthy complement to other compounding improvements in Monad including Asynchronous Execution, Optimistic Parallel Execution, and MonadDb. The rest of this article serves as a summary of how successive improvements in HotStuff have built upon each other, in order to explain the problem that MonadBFT solves. To summarize: 1. HotStuff gives us linear communication complexity so that we can have large validator sets, but it's not very efficient 2. Pipelined HotStuff gives us efficiency and low latency from proposing blocks every slot, but suffers from the problem of tail forks 3. MonadBFT gives us gives us tail-fork resistance and one-slot speculative finality --- HotStuff: Linear communication complexity enables large node counts HotStuff algorithms complete over the course of several rounds of communication, which generally take the form of “fan out, fan in” communication directly from leaders to validators back to leaders. Each round begins with the leader sending a message directly to other validators, who each send back a signed message attesting to having received the message. Provided that a supermajority (2/3) of the validators send back an attestation, each round ends with the leader aggregating the signed attestations into a Quorum Certificate (QC), which serves as proof that the supermajority attested to the previous message. HotStuff algorithms have multiple rounds of communication like this. - The first message from the leader is a block proposal - The second message is the QC for that block proposal - The third message is a QC about the previous QC (i.e. a QC-on-QC) - and so on If the procedure is interrupted at any time before finality, the block fails to finalize and is discarded; transactions from that block will have to be re-included in the next block. The original HotStuff protocol has no pipelining and has 3 rounds of communication before finality; the same validator plays the role of leader for each round. --- Pipelined HotStuff: A new block every slot raises efficiency Pipelining is what we all do intuitively when we have two loads of laundry to complete. Instead of waiting for load 1 to finish the full cycle before starting load 2, in pipelining we put load 1 in the dryer at the same time as load 2 goes in the washer. You can think of the original HotStuff as that naive approach to doing laundry (don't start on load 2 until load 1 is completely done), while Pipelined HotStuff is doing the intuitive behavior of progressing multiple laundry loads in a staggered fashion. In Pipelined HotStuff we stagger proposals, such that there is a new block proposed at each round, with the new block piggybacking on top of the message carrying the QC from the previous block. Block proposals march toward finality over the course of multiple rounds. The benefits of pipelining are significant. Pipelining raises the density of block proposals, since a block proposal is made in every slot, which raises throughput and lowers time-to-finality. However, there is one major drawback of pipelining, which is best illustrated with an example. Assume that the leaders for blocks N, N+1, and N+2 are Alice, Bob, and Charlie. If Bob misses his slot, then Alice’s proposal will be invalidated as well, because Bob’s message carries both his proposal and a QC for Alice’s proposal. When this happens, Charlie ends up being called upon to produce a block as if Alice’s proposal had never existed. We refer to this behavior as “tail-forking”, and it can be thought of as a mini-reorg of depth 1. The possibility of tail-forking has significant consequences, because missed slots aren’t necessarily by accident. If there is an opportunity to extract value by re-mining Alice’s block while re-ordering or omitting some of the transactions, then Bob and Charlie can collude to have Bob intentionally miss his slot, triggering an opportunity for Charlie to re-mine Alice’s block. This has been a significant drawback of Pipelined HotStuff protocols (some of which are in mainnet today). --- MonadBFT changes this MonadBFT is the first protocol to enable pipelining while making the algorithm tail fork-resistant. This tail fork-resistance comes from the fallback procedure when Bob misses his slot, which enables validators to piece together their collective knowledge of Alice’s proposal and its level of consensus within the validator set. In particular, under MonadBFT, if Bob misses his slot, then the fallback procedure has validators communicate to each other with signed attestations stating whether they saw Alice’s block. If the supermajority attests to Alice’s block, then Charlie is forced to re-propose Alice’s block. If Charlie wishes to propose a different block then he must provide a signed attestation from the majority of validators attesting to not seeing Alice’s block on time. In the typical case where Charlie re-proposes Alice’s block, he then gets to propose his block on the subsequent round. The result is two important properties: tail-forking resistance and speculative single slot finality. We have already spoken about tail-forking resistance, but let’s understand the impact on finality. As before, assume the leaders for blocks N, N+1, and N+2 are Alice, Bob, and Charlie. Under Pipelined 2-Phase HotStuff - i.e. before MonadBFT - as a validator (or a full node), you cannot finalize Alice’s block proposal until you see Charlie’s block proposal. Why? Because if you finalize as soon as you see Bob’s proposal, it is possible that Bob is messing with you by ONLY forwarding his proposal to you, and he actually plans to fail to send his proposal to everyone else, thus missing his slot. But in MonadBFT, as soon as you see Bob’s proposal, you can “speculatively” finalize Alice’s proposal because Bob’s proposal includes a QC on Alice’s proposal, which is proof that 2/3 of the network attested to Alice’s proposal. Even if Bob is messing with you by ONLY forwarding his proposal to you, and is going to end up missing his slot, you know that a supermajority of the network saw Alice’s proposal and, when they participate in the fallback procedure, will sign off on Alice’s proposal again. The only way that Alice's block won't get finalized is if the validators equivocate and sign off saying that they didn't see Alice's message. This fault is easily provable - we have signed conflicting messages from them. If the penalty for equivocation is substantial - and it should be - this “speculative” finality is actually not really that speculative. --- Takeaways MonadBFT is an extremely exciting development for consensus, and is a worthy complement to other compounding improvements in Monad including Asynchronous Execution, Optimistic Parallel Execution, and MonadDb. Huge congrats to @MohammadMJalal1 and @KushalBabel on this significant breakthrough. MonadBFT will be implemented shortly on Monad Testnet, which currently implements Pipelined 2-Phase HotStuff. For further reading, see the linked blog post and paper in the next tweet.

Please read this insanely good blog post from @category_xyz about RaptorCast - enabling efficient broadcast of huge blocks while retaining fault tolerance while also minimizing bandwidth reqs You'll learn a lot!