Pagination design

[frol]

Story

Explorer needs to represent tons of data and navigation is a crucial part of discoverability. Efficient search and pagination are the core of it.

Currently, we use infinite scrolling, which allows us to go progressively from the latest data to the oldest querying for the next page based on a cursor of the last item we fetched previously. While it works, it does not allow us to efficiently navigate through the historical data.

It is known that OFFSET is not an efficient way to implement pagination, and we are talking about 86k new blocks every day, shooting for 1k transactions per block (well, it is upper bound for a single shard and we are not going to stop there), so millions of transactions a day.

We need an efficient solution here and given we are solving Network Indexer (#236) issue, I figured to raise this issue earlier.

@SkidanovAlex @mikhailOK you have a great background in databases, maybe you can provide useful input here.

/cc @khorolets

[bowenwang1996]

While it works, it does not allow us to efficiently navigate through the historical data.

I don't think I fully understand the concern here. Are you referring to the time it takes to load new data?

[frol]

I am referring to the fact that there is currently only one way to explore data, that is by starting from the latest and scroll down the page, and it will load more and more data until it crashes a browser tab. We probably need some navigation with a timeline or classic pages (1, 2, 3, ...)

[khorolets]

@bowenwang1996 There is a concern about efficiency of offset in queries on tables where is a lot of data.

[bowenwang1996]

I see.

[FranklinWaller]

Could we reconsider this issue? Classic pagination makes exploring much easier. Personally, I believe it's best to mimic Etherscan as much as possible in terms of UX, making it easier for existing users to get familiar with.

[Zombieliu]

Yes friends, I submitted transactions in batches through scripts on the testnet, and the transaction information of the previous deployment contract was suppressed at the bottom. I did not specifically record tx hash and block hash. If it is similar to skyward, 800,000 transactions I'm thinking about which year I can turn to this deal hahaha

[think-in-universe]

thanks @Zombieliu

@Zombieliu here means that without pagination, it's impossible to search for the transactions executed several days or weeks ago, especially when a large number of transactions have run on the contract.