Bitcoin Mempool: Private Mempools

In a recent post on Mempool, I analyzed how transactions are propagated when different nodes in the network use different strategies to relay data to the mempool. In this article, I will examine the peculiarities of private mempools and their impact on the functionality of the public mempool, mining incentives, and the overall health of the Bitcoin network.

The main purpose of the mempool is to maintain aligned incentives between two key participants: miners and transaction users. Users are eager to execute transactions and are willing to pay miners a fee to do so. In turn, miners are interested in making a profit, and transaction fees serve as an additional source of income on top of the new coin subsidy in each block, and a necessary primary source of income in the long term as the subsidy dwindles.

Bitcoin is an incentive-based system. This basic dynamic governs the security of the system, where there are client(s) and provider(s), both seeking to satisfy their needs and desires, which ensures that the blockchain operates with a sufficient level of thermodynamic security.

Attempts to introduce friction into this simplification mechanism ultimately do not change the incentives of either party. A user who wants to make a particular transaction will still want to make it and will be willing to pay for it. A miner willing to accept such transactions will also be interested in including them in a block to earn a fee.

If the deal is confirmed, both parties will still have unmet wants and needs, and they will remain highly motivated to satisfy them in any way possible.

Miner API

Regular end users don’t always have the capital or skills to bypass the friction artificially created between the two parties, but miners certainly can. As the famous saying goes, “if you build it, they will come.”

The ideal situation for miners is to receive transaction fees through a public mempool. This means minimal overhead, as it is enough to run a standard Bitcoin client, which provides a reliable distribution mechanism and a high probability of miners receiving the most profitable transactions without any additional effort. Just install the client and run it.

However, under extreme conditions, such as when the network attempts to filter out agreed-upon transactions as they are transmitted, this traditional assumption may be called into question.

In this case, miners have every incentive to implement out-of-band mechanisms to accept transactions that are not correctly transmitted over the network. An example of this approach is Marathon's Slipstream API for non-standard transactions. In fact, there is a long-standing practice that emerged almost a decade ago that was widely adopted by many mining pools and continues to exist today – transaction accelerators.

We are now in the era of Full-RBF, where any transaction, regardless of whether it has the historical “opt-in” flag, can have its fee increased. Any node that has upgraded to Full-RBF will relay any transaction using an unconfirmed output pending in the mempool if it offers a higher fee. This was not always the case. Historically, only transactions that were initially flagged to opt in to RBF could be replaced and were expected to propagate through the network.

Transaction accelerators were developed by miners to simplify this behavior for transactions that did not opt in to RBF.

Third Party APIs

While the cost of creating your own transaction submission API for a miner or pool is not too high, it is not a free endeavor. It still requires at least one developer and time to design and implement any software. While the cost curves are not prohibitive, they still favor larger miners over smaller ones in terms of the resources they have to devote to such a project.

Mempool.space has demonstrated that creating such an API by third-party developers unaffiliated with miners is a viable solution, allowing miners to simply connect to their service instead of going through the effort of building it themselves. However, this has its drawbacks: such a third party will not provide and maintain such a service for free. They will want their share of the profits.

There are two possible directions for this dynamic to go: either these services will eventually require higher costs to provide revenue for both miners and service providers, or miners will have to share a smaller share of revenue to keep these services competitive with those run directly by miners. This means that miners using a third-party API to send will receive less revenue than those using their own API.

Private order flow

Any of the above possibilities raises serious issues regarding overall system incentives, the trustworthiness of end-user software, and perhaps even the security models of second-layer systems that rely on the use of pre-signed transactions and reactive security models to protect user funds.

When transactions are sent through a private API, they remain invisible to network participants until they are confirmed in a block. The entire queue of unconfirmed transactions using these systems remains opaque. This can be done by public operators of such APIs, but not on a trustless basis. There is no way to prove or guarantee that the operators are not hiding information.

Hiding transactions from public view can distort estimates of fees that users make, and even opens up the possibility of manipulating those fees by stuffing blocks.

Source: cryptonews.net