Open sijie opened 6 years ago
It would be great for stagare usecase, like reading a batch of sequential entries woth random access pattern (opposite to tailing reads)
A new wire protocol rpc would great as well
FEATURE REQUEST
We create continuous streams of growing data, eg ledgers with entities. However, we also require random access in the underlying data stream.
Proposed API's:
Returns the byte offset of the last byte of the last entry committed to the ledger.
Reads all bytes between startOffset and endOffset (inclusive), returned per stored entry. In case the endOffset is beyond the end of the ledger, the behavior should be the same as readEntries.
Reads all bytes from startOffset to current confirmed end of ledger, returned per stored entry.
Currently using a different solution to tackle this use case, with less durability/scalability/etc. It would make our future architecture simpler and better overall.
While not needed for our use-case, to be API complete, uncommitted API's might be good to have.
While not immediately needed for our use-case and we can tackle this with other polling mechanisms, it would be useful if we can open read binary from a ledger. Starting at a given startOffset, keep receiving byte[] until either the handler is closed or endOffset is met.
FEATURE REQUEST
Currently in a ledger, we indexed entries by entry id. It would be good to have an index by offsets. This allows supporting APIs like:
nice-to-have
entry(/request) oriented api is not very good friendly to resource-usage when do prefetching or batching reads. offset oriented api is much better for estimating resource usage.