paritytech / polkadot-sdk

The Parity Polkadot Blockchain SDK
https://polkadot.network/
1.83k stars 667 forks source link

Get rid of the distinction between Parathreads and Parachains in the scheduler #667

Closed eskimor closed 1 month ago

eskimor commented 1 year ago

We still differentiate between parathreads and parachains in the scheduler, which is now obsolete.

Reasons we do are twofold:

  1. A parathread provides a CollatorId, which parachains don't.
  2. Retry logic: We special case parathreads and only retry them not parachains.

Resolving 1:

No need to differentiate between parathreads and parachains here, we can just have a struct for both:

struct CollatorRestrictions {
   /// Collators to prefer/allow.
   /// Empty set means no restrictions.
   collator_peer_ids: Set<OpaquePeerId>,
   restriction_kind: CollatorRestrictionKind,
}

/// How to apply the collator restrictions.
enum CollatorRestrictionKind {
  /// collator_ids and peer ids mentioned will be preferred in connections, but others are still allowed.
  Preferred,
  /// Any collator with a `PeerId` or `CollatorId` not in the sets of `CollatorRestrictions` will be rejected.
  Required,
}

This representation is not only useful for parathreads, but also parachains. While parathreads will very likely only use CollatorRestrictionKind::Required, parachains can use Preferred to keep the set open, but still guarantee some known "good" collators can connect no matter what.

Resolving 2:

The assignment provider provides a function like:

fn get_num_retries_for_core(CoreId) -> u32;

With this the scheduler just asks the assignment provider how many retries (retries apply to availability timing out) should be accomplished for a given core. Again no need to differentiate between parathreads and parachains.

eskimor commented 1 year ago

If we can get some representation of PeerId into the runtime we should even get rid of collator_ids ... let's keep it simple. I don't think we need them.

bkchr commented 1 year ago

We already have OpaquePeerId. However, we have for everything unique keys. So, we should probably keep collatorids for signing the povs.

eskimor commented 1 year ago

Yes, PoVs will still be signed by the CollatorId - although I yet have to understand what the value is, given that CollatorIds are meaningless on the relay chain: How is checking the signature adding any value? Anyone can create some CollatorId and sign with the corresponding private key. We are never verifying that the signing collator has anything to do with the para (which is by design, because we wanted the set to be open) - nevertheless, why check a signature for some random id someone provides to us? I must be missing something?

bkchr commented 1 year ago

Haha, I was not that sure myself that we need CollatorId. One thing for sure is that if I buy a slot for my collator, I also want to be able to be the only one sending the data. At some point we were also discussing stuff that you send PoVs via one or multiple hops to the validator and then you can not rely on the PeerId sending the data.

burdges commented 1 year ago

Did we ever choose the economics of buying parathread slots? In principle parathreads might desire some control over this aspect, not merely restrictions upon making blocks. It'd make look like this:

You supply a CollatorId or whatever when you buy parathread blocks, which always occurs in batches of size n = min_batch_size + k min_batch_increment (parathreads set those parameters). The parachain state on the relay chain has an acknowledged_parathread_blocks count, but the purchase itself only appends (CollatorId,n) to a tentative_parathread_blocks pool. Any parachain block could "acknowledge purchases" by removing entries from tentative_parathread_blocks, which increases acknowledged_parathread_blocks by their sum. Anytime the parachain makes a parablock then tentative_parathread_blocks is decremented but we require tentative_parathread_blocks >= 0 after the parablock occurs.

A parachain's own logic determines how CollatorId restricts making future blocks as well as how "acknowledging purchases" changes these restrictions, but in principle if tentative_parathread_blocks.len() > 0 then some collators could make a block, even if acknowledged_parathread_blocks=0.

This enables diverse schemes schemes like:

It's likely acknowledged_parathread_blocks eventually winds up larger than reachable block production tickets on the parathread, like if someone buys many blocks but looses their keys, so parathreads should've logic that harvests excess very slowly.