Timestamp for "pin creation" event

[lidel]

Why do we need this?

Pin "creation" timestamp often comes up while discussing pagination and listing filters the order in which returned pins should be returned. @obo20 noted that without having means of sorting in the spec, different pinning services may create contradictory behaviors.

I am leaning towards opinion we should add it as mandatory field to PinStatus object, and enable filtering based on "creation" date.

It will enable:

• api clients to fetch only recent pins
• pinning services to optimize backend (eg. by creating time series / indexes)
• automatic sorting of returned results and deterministic pagination behavior across vendors (details TBD, depend on how we resolve this issue first)

What do we mean by "pin creation"?

When we think about pin lifecycle, there are two "creation" events:

1. Pin operation is queued, waiting to be processed
2. Pinning is finished, data is pinned

This introduces question:

Q: should we add one or two fields?

Any ambiguity would go away if we add both (PinStatus.queued and PinStatus.pinned), however I am unsure how useful queued is after reaching pinned status. Storing unused value feels like a waste.

If we add a single field (PinStatus.updated) should it be immutable moment of pin request entering the service (timestamp of queued event), or should it be updated when reaching pinned status?

Thoughts? Is there a better way of representing this? (mind we want to keep API minimalist)

cc @obo20 @GregTheGreek @priom @jsign @sanderpick @andrewxhill

[Gozala]

If we add a single field (PinStatus.updated) then we could store the timestamp of queued event until reaching pinned status.

Mutating timestamp seems like bad idea to me, that may lead to some nondeterminism.

Any ambiguity would go away if we add both (PinStatus.queued and PinStatus.pinned), however I am unsure how useful queued is after reaching pinned status. Storing unused value feels like a waste.

Do not know if this a good idea, but you could encode second value not as timestamp but a delta (from pin request start to completion)

[lidel]

Indeed. We could remove nondeterminism by using the moment pin request entered pinning service as immutable timestamp, and then base pagination on it. It should make it easier to have same behavior across different vendors.

I opened PR #39 that adds mandatory PinStatus.created. Unsure if the time of reaching "pinned" status is useful to end users, so I've skipped it for now.

[lidel]

Some additional musings on why "pinned" time is not useful for billing:

On the first thought knowing the "pinned" time on top of "created" time sounds like useful distinction. One could say, that pinning service could bill user only for the time entire dataset was being provided, excluding the time it took to fetch missing blocks.

Sadly that would enable various types of abuse:

• freerider could create a DAG that is always 99% fetched, never paying for the pin
• malicious actor could flood pinning queue with pin requests for DAGs that are only 1% routable, jamming the queues and degrading service for regular users

By billing from the initial request, above abuse is financially disincentivized, and users are incentivized to take advantage of providers hints to speed up initial transfer.