Batch Operations

[anlumo]

Is your feature request related to a problem? Please describe.

My data source allows fetching multiple elements at the same time, and I have to display a list of 50+ entries. Right now, these 50 fetches can't be batched together to a single request in Stock.

Describe the solution you'd like

Two new functions .getList and .freshList on Stock that take a list of Keys instead of a single one. This information should then be passed on to the Fetcher and the SourceOfTruth as-is. If the old .get or .fetch is used, a list with a single entry could be supplied instead (so there's still only a single function to be called).

Describe alternatives you've considered

It's possible to debounce the Fetcher and SourceOfTruth since they're async, and so collect all requests coming in within a certain timeframe, collect them, send a single request, and then split up the result again. However, that adds lag to the whole operation that shouldn't be necessary and is probably also pretty complicated to implement.

[mirland]

Hi! The Key is a generic type, so in your case Key could be a List<int>. In that case the fetcher will be called with a list of ints, then in your fetcher you have to fech your data.

typedef DataSource<T> = Future<T> Function(List<int> keys);
void createStock<T>(DataSource<T> dataSourceMethod) {
  final fetcher = Fetcher.ofFuture<List<int>, T>(
    (key) => dataSourceMethod(key),
  );
  final stock = Stock<List<int>, T>(
    fetcher: fetcher,
  );
}

This resolves your issue, right?

[anlumo]

But the list of ints isn't always the same. What if the Source of Truth has a partial match, and the rest of the items have to be fetched?

[mirland]

But does your fetcher support multiple requests at the same time? Or do you want to call the fetcher multiple times at the same time with different keys?

I think you have a "simple" fetcher (like the Twitter fetcher example), that fetches a page by an id. However you need to fetch multiple pages simultaneously, Am I right?

If that is the case, you need to use RX to merge the partial results.

If it's not your case. Can you give me an example displaying your "interfaces"?

[anlumo]

But does your fetcher support multiple requests at the same time?

Yes.

However you need to fetch multiple pages simultaneously, Am I right?

Yes.

If that is the case, you need to use RX to merge the partial results.

The problem with that is that it's a lot of work. Information is discarded at the stock API level and then has to be reconstructed on my side again with added latency (due to the debounce delay).

[mirland]

We discussed and we think this case is not a common case and it's not in our backlog right now. I think you can implement using some RX without delay. If you provide us the data source "interfaces" (abstract class) we can help you with that.