deephaven / deephaven-core

Deephaven Community Core
Other
250 stars 80 forks source link

Java client should support batching queries #4639

Open rbasralian opened 11 months ago

rbasralian commented 11 months ago

The API for the Java client should allow multiple queries do be submitted at once (e.g. call .snapshot() with an array of TableSpecs).

It should be possible to have them either run synchronously or concurrently. Also it should be able to have them execute either serially or in parallel at the server.

So if you do something like .snapshotParallelAsync(array of 100 TableSpecs), you should get back 100 Futures/have some kind of callback when they are completed (which may be in a different order than they were submitted in).

devinrsmith commented 11 months ago

I'm not sure this level of client implementation orchestration around synchronous vs concurrent is something we want to provide.

I do think much of want you want could be accomplished if the java client provided a better / more asynchronous API.

For example, we could provide at the low-level:

public interface Session ... {
    ...
    CompletableFuture<TableHandle> executeAsync(TableSpec table);

    List<CompletableFuture<TableHandle>> executeAsync(Iterable<TableSpec> tables);
}

And at the higher level:

public interface BarrageSnapshot ... {
    ...
    CompletableFuture<BarrageTable> entireTableAsync();
}

and in this way, the client could orchestrate the asynchronous execution of a lot of snapshots from a single thread.