[feature request] more JSer-friendly Collector for Lumos

[homura]

Motivation

As a JS developer, I'm actually more comfortable using functions such as map, filter, reduce, etc. to handle Iterable rather than for...in / for...of. And the AsyncIterator exposed by Lumos is much more uncommon. i'm more used to using Promise for async

https://github.com/nervosnetwork/lumos/blob/d5e7636736e22c3247d36b4160213e74ed23dd0e/packages/ckb-indexer/src/collector.ts#L284

How

Add a new method takeWhileAggregate for Collector like this

declare class CellCollector {
  takeWhileAggregate<T>(
    initialValue: T, 
    aggregate: (aggr: T, cell: Cell, currentIndex: number) => T, 
    takeWhile: (aggr: T, cell: Cell, currentIndex: number) => boolean,
    // defaults to false. If true, exclude the last takeWhile's `() => false` collected element
    excludeLast?: boolean,
  ): Promise<T>;
}

Examples

Collect Lock-only Cells and Calculate Total Capacity

const total = await new CellCollector(...).takeWhileAggregate(
  BI.from(0),
  (sum, cell) => sum.add(cell.cellOutput.capacity),
  () => true,
);

Collect 10,000 Unit of sUDT Cells

const { cells, amount } = await new CellCollector(...).takeWhileAggregate(
  { cells: [] as Cell[], amount: BI.from(0) },
  ({ cells, amount }, cell) => ({
    cells: cells.concat(cell),
    amount: amount.add(Uint128.unpack(cell.data)),
  }),
  ({ amount }) => amount.lt(10000)
)

if (amount.lt(10000)) {
  throw new Error('not enough sUDT, expected over 10000, actual ' + amount )
}

[IronLu233]

I have some questions for the API declare:

Is "Drop last cell in result" a very common demand for CKB developer? I think let the API invoker do the "Drop last" work is better. For example, an lodash initial method can do this work.

And about TakeWhile and aggregate parameters. seems can merge two high order function into one.

type Reducer = (acc: T, current: Cell, index: number) => ({  value: T, hasNext: boolean/* If it's false,  the collect process will finished */ })

[homura]

Is "Drop last cell in result" a very common demand for CKB developer?

for the UTxO model, the input of the general tx is required to over-collect the UTxO, so 'drop last cell' is uncommon, the excludeLast here is inspired by RxJS's takeWhile. But the excludeLast in API is unintuitive, and i now feel it is unnecessary

And about TakeWhile and aggregate parameters. seems can merge two high order function into one.

the reduce(fold/aggregate) and take_while is common in many languages' iterator API, we should ensure that the API is common, another reason for splitting the reducer into takeWhile and aggregate is that it is it can work more easily with other (higher-order) functions

collector.takeWhileAggregate(
  BI.from(0),
  ...,
  Ramda.always(true)
)

[IronLu233]

the reduce(fold/aggregate) and take_while is common in many languages' iterator API, we should ensure that the API is common

I agree this point. Keeping it's same as common functional programming library API.

My next concern is how to tell our user to use it correctly. So I suggest add a example, and use tsdoc anotation for each parameter.