Add Secondary Market Functionality to Broker Pallet

[poppyseedDev]

After extensive deliberation and discussion at Lastic, we have concluded that integrating a secondary marketplace feature directly into the broker pallet is the most efficient approach. Given the limited scope of this addition and the minimal security risks involved, incorporating this functionality into the broker pallet is both practical and effective.

An alternative method could involve creating a separate chain with XCM functionality to handle core transfers and listings. However, maintaining such a solution would introduce significant financial and development overhead. For a feature of this scale, adding three functions directly to the broker pallet is a simpler and more sustainable solution.

[Szegoo]

There are a couple of reasons why I believe this isn’t a good idea. One might argue that I might be biased because I am working on a project that, among other things, addresses the issues mentioned in this RFC. However, I will do my best to genuinely review this RFC. The following list outlines why I don’t consider implementing marketplace functionality into the broker pallet to be a good idea:

1. It is important to consider who actually needs Coretime in the first place. Currently, the consumers of Coretime are parachains. Many of these parachains share the goal of being decentralized, self-governing entities. To achieve this goal, a parachain cannot depend on a specific group of people for Coretime procurement. Adding functionality to the broker pallet that resembles a regular NFT marketplace is not the ideal solution. As mentioned, parachains are transparent, autonomous entities. Due to their transparency, they may encounter issues when interacting with the marketplace described in this RFC. One of the main problems is front-running. Since parachains are transparent, meaning anyone can foresee their market actions, there is a risk that someone could purposefully purchase the region the parachain is looking to procure just before it does. This could leave the parachain without any Coretime, thereby halting its operations. A possible solution could be for the parachain to rely on a set group of people for Coretime procurement, but this would go against the goal of decentralization.

2. The user who listed the region for sale can unlist it at any time. This means they can sabotage a parachain that intends to purchase that specific region just before it does so. Again, this could leave the parachain without any Coretime, thereby halting its operations.

3. Would we allow the trading of regions that are 'active' during the current bulk period (i.e., region.begin < last_committed_timeslice)? Since the RFC doesn’t mention this, I would assume it is inherently allowed. However, trading these regions on the marketplace described in this RFC would have some issues. These regions are gradually losing value because their potential to be used is wasted by having them listed on the market instead of being assigned to a specific task. These regions shouldn’t have a fixed price; instead, they should have a dynamic price that decreases with each timeslice. This is what we are implementing in the RegionX Coretime marketplace: https://regionx.gitbook.io/wiki/advanced/dynamic-coretime-pricing

The rationale for implementing the described functionality on the Coretime chain is not really convincing to me. The primary argument is that this approach is simpler than implementing it on a separate parachain. However, given Polkadot’s multi-chain architecture, we should leverage its structure along with technologies like XCM to develop multi-chain protocols and applications. We should not hesitate to distribute functionality across multiple chains where it makes sense, which, in my honest opinion, is the case for Coretime marketplaces.

[phillux]

1. It is important to consider who actually needs Coretime in the first place. Currently, the consumers of Coretime are parachains. Many of these parachains share the goal of being decentralized, self-governing entities. To achieve this goal, a parachain cannot depend on a specific group of people for Coretime procurement. Adding functionality to the broker pallet that resembles a regular NFT marketplace is not the ideal solution.

The fact that agile coretime is tradable and accessible to those beyond simply builders is a necessary attribute to it being "agile." This is a benefit to the chain builder as much, if not more so, as it is to the non-builder. There will always be an inherent advantage to a builder who utilizes the coretime for securing and validating a chain over a trader who cannot do anything with it besides resell it. The fact that cores are NFTs make a secondary marketplace possible, and something that has been in the works for thinkers in blockspace since before agile coretime's introduction. (ctrl+f=secondary)

The ideal solution is to make trading as seamless as possible, and the building for competitive secondary coretime markets as simple as possible. This helps the builder who wants to trade their excess coretime just as much as the non-building trader.

One of the main problems is front-running.

Front-running is out of the scope of this RFC as this RFC does not enable, nor protect users from, front-running. Front-running protections can be established in other ways if desired, such as steeper lead-in periods. It could be argued that front-running could be beneficial to the market health of polkadot which does not have any value-extracting mechansims, dissuading a large part of the greater crypto market from participating in Polkadot. At the end of the day, front-running is a result of high-demand for high-value transactions, and would be a good problem to have in the future and "solve" or sort out in a future RFC.

If on-demand coretime is available, there is no halting of any parachain's operations.

A possible solution could be for the parachain to rely on a set group of people for Coretime procurement, but this would go against the goal of decentralization.

I would argue that crowdloans were precisely "parachains relying on a set group of people for coretime procurement," and this has nothing to do with decentralization.

2. The user who listed the region for sale can unlist it at any time. This means they can sabotage a parachain that intends to purchase that specific region just before it does so. Again, this could leave the parachain without any Coretime, thereby halting its operations.

Again, if on-demand coretime is available, there is no halting of any parachain's operations.

3. Would we allow the trading of regions that are 'active' during the current bulk period (i.e., `region.begin < last_committed_timeslice`)? Since the RFC doesn’t mention this, I would assume it is inherently allowed. However, trading these regions on the marketplace described in this RFC would have some issues.
   These regions are gradually losing value because their potential to be used is wasted by having them listed on the market instead of being assigned to a specific task. These regions shouldn’t have a fixed price; instead, they should have a dynamic price that decreases with each timeslice. This is what we are implementing in the RegionX Coretime marketplace: https://regionx.gitbook.io/wiki/advanced/dynamic-coretime-pricing

A simple descending cost model makes sense and is trivial to implement, aside from being obvious.

The rationale for implementing the described functionality on the Coretime chain is not really convincing to me. The primary argument is that this approach is simpler than implementing it on a separate parachain. However, given Polkadot’s multi-chain architecture, we should leverage its structure along with technologies like XCM to develop multi-chain protocols and applications. We should not hesitate to distribute functionality across multiple chains where it makes sense, which, in my honest opinion, is the case for Coretime marketplaces.

Since we are not talking about adding functionality to the relay chain, I fully agree that we should leverage the parachain structure, and in this case we are by adding functionality to the coretime PARACHAIN. System parachains are parachains, and it's up to the community what they should provide. It is in the community's wishes to stop further fragmentation and adding useful functionality to system parachains in an unbiased way makes sense.

One might argue that I might be biased because I am working on a project that, among other things, addresses the issues mentioned in this RFC.

Yes, one might argue this and I definitely believe that you are biased because this RFC would render much of your work outside of your sole control. We believe that Polkadot needs to be easier to develop on for its continued growth, and adding functionality to system chains is the easiest way to encourage growth and reduce the amount of technological gatekeeping that tends to occur with parachains feeling threatened by functionality that helps everyone.

We need to understand that Polkadot is changing with JAM on the distant horizon which threatens the entire parachain model, and continuing a conservative approach will cause a split between JAM and Polkadot that we cannot afford. We must continue to adapt in order to maintain relevance.

That being said, we look forward to feedback on improvements to this RFC from stakeholders without clear conflicts of interest.

[poppyseedDev]

To add onto what @phillux metioned, firstly I believe that emphasizing the efforts made towards other solutions (by both Lastic and RegionX) is unproductive if those solutions are less optimal than what this PR proposes.

Building a separate parachain to handle this functionality is inefficient and impractical. The only financial gains would be from secondary Coretime sales. However, to establish and sustain this parachain, significant costs are involved:

• Core Purchase: You would need to buy a region on both Kusama and Polkadot, with only around 50 cores available in total.
• Maintenance Costs: You need to renew the region monthly, which is financially burdensome.
• Revenue Generation: To cover costs, you would need to resell approx. at least 50 regions each month, which is unlikely. (taking a 2% cut from each sale, would cover you only the costs of renewing a region)
• Development and Operational Costs: There are additional costs that you wouldn't be able to cover which include development, maintenance, indexing, and other operational expenses.

Therefore, such a chain is financially unsustainable and would require Treasury funding, which is unfair to the community. Additionally, developing a new chain for this functionality would take several months, whereas adding three functions to the broker pallet can be done in a few days.

Moreover, creating a chain solely for this functionality would centralize the secondary marketplace, potentially excluding certain actors from participation. This contradicts the principles of decentralization. Building extended functionality on another system chain also lacks economic sense.

Addressing specific concerns:

1. Decentralization and Standardization: Integrating the marketplace directly into the broker pallet enhances decentralization for parachains. It provides a standardized and transparent mechanism for all parachains to engage in Coretime trading.

2. Trading of Active Regions: Allowing the trading of regions that are 'active' during the current bulk period is manageable. Users can adjust pricing at any time, and in my opinion linear pricing curve should not be imposed on them as proposed.

3. Separate Chain Option: This proposal does not preclude the option of creating a secondary chain that communicates with the Coretime chain via XCM. Users can still transfer regions over XCM and sell them on another chain if they choose.

In conclusion, integrating the secondary market functionality directly into the broker pallet is more efficient, cost-effective, and aligned with the principles of decentralization. This approach reduces overhead, speeds up implementation, and maintains flexibility for future enhancements.

[Szegoo]

If on-demand coretime is available, there is no halting of any parachain's operations.

This would require the parachains to dynamically switch from bulk to on-demand coretime. Solving this appropriately wouldn’t be as easy as you described.

Front-running is out of the scope of this RFC

It is not. It is important to consider how projects will actually procure coretime. If the team behind the parachain handles the coretime procurement, the issue of front-running doesn’t exist. However, if the parachain’s governance handles the procurement, front-running becomes an issue. For example, everyone can track the governance decisions of the parachain. If someone proposes to buy a region from the market, by the time it passes, the seller could have unlisted it, or someone else could have bought it. This system would be very fragile.

I would argue that crowdloans were precisely "parachains relying on a set group of people for coretime procurement,"

It really isn’t. It is called a crowdloan for a reason. Anyone who supports the project can contribute by allocating some of their own tokens to help it procure coretime (i.e., win the slot auction). It isn't dependent on the parachain team to finance the coretime procurement; instead, it is community-driven.

We need to understand that Polkadot is changing with JAM on the distant horizon which threatens the entire parachain model

Parachains won’t be removed with JAM. There will still be a ‘parachain service’.

I definitely believe that you are biased.

There is a reason why I am developing a project in a particular way. I have considered adding this functionality to the Coretime chain as well, but I decided against proposing it because I believe it is not the right approach for the reasons I mentioned.

[Szegoo]

@poppyseedDev Thanks for the writeup. However, I think most of this is off-topic for this RFC. We can discuss it elsewhere.

To answer your main concerns in short: No, the treasury won't have to pay for the Coretime procurement and other costs associated with running the parachain. Also, DOT token holders will vote on the parachain's code upgrades.

If you have further questions or concerns about the parachain, we can discuss them somewhere more appropriate.

[eskimor]

My two cents: First of all I do think that having secondary markets live right on the Coretime chain would make perfect sense and this is also why I would love to see smart contracts enabled on Coretime. Until we have that, indeed the next best solution would be to include them in the runtime directly, but first of all I don't think that should go into the broker pallet, for at least a couple of reasons:

1. The broker pallet is already involved enough.
2. Secondary markets are certainly implementable in different ways, as evidenced by the discussion on this very ticket, making a biased default implementation right in the broker pallet quite questionable.
3. Maintainer of the broker pallet is currently Parity. The idea of secondary markets, was to be offered by the community, not by Parity. I don't see how maintenance will not end up being the responsibility of Parity, when integrated into broker directly.

What could work is that teams wanting to offer secondary markets provide additional audited and reviewed pallets that can be deployed on the Coretime chain. Those pallets should be named after the team maintaining them, e.g. region_x_coretime_market/lastic_coretime_market, ....These pallets could then also introduce their own token or fee system, so that the teams maintaining the market get the necessary funding.

There are benefits and downsides to this approach. On the upside, you don't need to run your own parachain, which means less cost, less complexity and better efficiency. On the downside: Less flexibility, slower iteration: The runtime needs to be trusted, hence any changes you make will need to be approved by the fellowship. So I would say this is only viable if those pallets stay small and will not evolve much. And obviously as this is a significant maintenance burden on the fellowship, it is also up to the fellowship to decide whether this is an option at all. In any case this does not scale too well, so mid term we should have smart contracts. Also as there would be maintenance burden on the fellowship it is not clear how this would go together with a for-profit system.

TL;DR, my opinion: Secondary markets on the coretime chain: Yes. In the broker pallet: No. Directly in the runtime: Maybe a reasonable intermediate solution until we have contracts.

[burdges]

What could work is that teams wanting to offer secondary markets provide additional audited and reviewed pallets that can be deployed on the Coretime chain.

Another benefit is simply the precedent of parity using this model somewhere. It's a great model for many things in the ecosystem outside of the core functionality of polkadot.

[poppyseedDev]

Thanks for the feedback. How should we proceed with this issue?

Does it make sense to close this PR and instead propose adding smart contract pallets to the Coretime chain? This could allow us to implement the secondary marketplace functionality more flexibly in the future. In the mid-term, we could add some basic functionality as a specific extension pallet, which would serve as an intermediate solution.

Or, do you think it still makes sense to keep this issue open and present it as is?

Your guidance on the best path forward would be greatly appreciated.

[burdges]

instead propose adding smart contract pallets to the Coretime chain?

"Audited" often precludes smart contracts, maybe folks want that eventually, but afaik they're all in flux now anyways. I'd focus upon (a) messaging endpoints that addressed the initial replies concerns, albeit with some latency, and (b) some really basic secondary market functionality.

Afaik, we do not need anything complex like auctions or real matching here, maybe sell above some price, and buy below some price, with the buy choosing the oldest matching sell, or even the bbuy containg a list of acceptable ones, but buys can fail if all below that price or on the list disappear first. If you need the list, then the list could be encoded as a merkle tree, so then the collator can shrink the tx to just the sell it picks. @jonasW3F might've optinions.

[eskimor]

I think an RFC (or a wish for change) proposing to add smart contracts to the coretime chain and let the fellowship/token holders decide, certainly makes sense. You should also be aware of this one, currently having huge support. It would make perfect sense to have a wish for change to also add pallet-contracts to the coretime chain, while we are at it. This will not happen until end of year though, if we want something faster, we would need to do Frontier, with the caveats in referendum 885.

For adding a pallet as an intermediate solution: I think this can be proposed to the fellowship. For pursuing this one: I am skeptical it will find support. For the additional pallet solution: I don't know, could go both ways. Contracts, given how 885 is going so far, might be the safest bet, but probably also the one with the longest timeline.

[burdges]

Afaik AssetHub lacks our locks, staking, slashing, and governance, which makes 885 make more sense. Afaik we've never interfaced smart contracts directly to those pallets before, so maybe 885 should look somewhat like messages anyways.

After extensive deliberation and discussion at Lastic, we have concluded that integrating a secondary marketplace feature directly into the broker pallet is the most efficient approach.

Why not some message interface for the broker pallet? It already has some transfer function, right?

[poppyseedDev]

Why not some message interface for the broker pallet? It already has some transfer function, right?

@burdges would you mind expanding on this?

I have expanded on the proposal to include the possiblity of having this be an extension pallet that would live on the Coretime chain.

[burdges]

We'll anyways have the broker pallent recieve some messages, so then a market functionality could live on multiple other paracahins, right? Assuming this, what advantages does having a market on the same chain bring?