Measure ingest latency within indexer node

[masih]

It would great if we can measure the ingest latency within storetheindex, defined as:

• The elapsed time, starting from the time at which an announcement is accepted by an indexer node, to the time at which the advertisement corresponding to that announcement is marked as processed.

[gammazero]

When an announce is accepted, the indexer will attempt to begin a sync. If there is already a sync in progress, then the pending sync is queued, waiting for the previous to complete. Should this metric measure this waiting time, or should the time start when the sync for that ad actually begins?

[masih]

It should include the wait time.

Context This metric is measured as part of an end-to-end ingest latency as it occurs for a user of network indexer: the elapsed time starting from the moment an ad is published to the movement multihashes of that ad are found via finder API.

Since we cannot accurately measure the total end-to-end time, as it depends on announcement propagation etc., this issue is about measuring the portion that falls within our control: how long did it take a node to ingest an ad once it learnt about its existence. Sync wait is an implementation detail that an end user need not to care about.