Had a few questions regarding achieving server pagination and counting messages in a KTable. They are closely related to each other which is why instead of opening couple issues I have put them in a single one for brevity.
Is it possible to achieve full server pagination when doing a pull query on a materialized view? I have tried something like the following
context.CreatePullQuery<JsonElement>(viewTableName)
.Take(pageSize)
.Skip(pageNumber) // this doesn't seem to be allowed
.GetManyAsync()
.ToListAsync();
which doesn't let me define the Skip parameter on the server. If I place the Take and Skip after the GetManyAsync it just does the pagination after it polls all the data. I haven't found a Offset feature in the .NET client, so I was wondering if there is some other way?
In addition to the first question, from the documentation I've read it states that there is no OrderBy supported because of possibility of records being on different partitions, but it was an older thread, so I was wondering are there any changes in this? What if we are dealing with a single partition topic?
Probably one solution would be when creating the source table to do the ordering, but what if we need dynamic ordering?
In order to return pagination metadata, we will need to have a total count of the messages in the KTable. One way I managed to get it to work is the following:
I was wondering if there is a more efficient way to get the (filtered) data and the total `Count` in one call?
4. Last thing, is there a way to **see what is the ksql query that is being executed behind the scenes when we are using the .NET `KSqlDbContext`**? Do we always need to make sure that we create an `IPullable` and then do a `.ToQueryString()`?
Thanks in advance.
Hi,
Had a few questions regarding achieving server pagination and counting messages in a KTable. They are closely related to each other which is why instead of opening couple issues I have put them in a single one for brevity.
Is it possible to achieve full server pagination when doing a pull query on a materialized view? I have tried something like the following
which doesn't let me define the
Skip
parameter on the server. If I place theTake
andSkip
after theGetManyAsync
it just does the pagination after it polls all the data. I haven't found aOffset
feature in the .NET client, so I was wondering if there is some other way?In addition to the first question, from the documentation I've read it states that there is no
OrderBy
supported because of possibility of records being on different partitions, but it was an older thread, so I was wondering are there any changes in this? What if we are dealing with a single partition topic? Probably one solution would be when creating the source table to do the ordering, but what if we need dynamic ordering?In order to return pagination metadata, we will need to have a total count of the messages in the
KTable
. One way I managed to get it to work is the following:// make the second call to retrieve the data