search

ipni / specs

:book: Technical Specification of IPNI Protocols
11 stars 2 forks source link

Add SeqNum field to advertisement specification #30

Open gammazero opened 7 months ago

gammazero commented 7 months ago

Do you think we need a separate doc for this?

Does not seem necessary, since this only adds information and does not change protocol behavior. I will add a new subsection to the doc, and then we can decide if a separate doc is warranted.

There are a few edge cases to expand on, e.g.:

  • what should new providers do.

Use sequence numbers. (added to spec)

  • what should existing providers do.

Use sequence numbers (added to spec)

  • what does it mean to reset the sequence number of allowed at all.

Reset means set back to 0, allowed when there are 9007199254740992 advertisements on the chain. (clarified in new subsection)

  • how it would impact the advertisement signature and its verification.

If present then it is included in the data that the advertisement signature is calculated over. (in doc)

  • how should indexers interpret its presence, validation, etc.

Indexers are not required to validate that SeqNum is monotonically increasing, but an incorrect sequence number may result in incorrect indexing information or may be ignored resulting in some indexing features being unavailable for that chain's provider. If an indexer depends on the sequence number for more than display and reporting purposes, it can choose not index advertisement chains that do not have a monotonically increasing SeqNum.

A separate doc would also make it clear what's current and what's being added, similar to how IPFS IPIP process functions.

No behavior change, only additional info, so seems like new doc not needed. However, if an IPIP process is the way we want to handle all protocol updates, then we can do it that way.

masih commented 7 months ago

Thank you for including the PR comments in the document.

Indexers are not required to validate that SeqNum is monotonically increasing

My sense is that flexing on this significantly reduces the effectiveness of Sequence numbers.

IPIP process

It seems like we are moving towards a world where there are different versions of IPNI spec, and indexers/providers that comply with different versions. This is not a blocker for now but I think we should think about introducing some kind of versioning to allow a user to infer what they can expect from an indexer instance. For example, assuming we make the validation of sequence number strict then another indexer can rely on it to concretely infer their lag in ingestion. They can also infer that some other indexer that does not support the newer spec cannot be relied upon to offer such insight. This allows backward compatibility while at the same time offer meaningful usefulness in exchange for protocol changes.