feat(arena): optimize data frame deserialization

flock-lab / flock

Flock: A Low-Cost Streaming Query Engine on FaaS Platforms

https://flock-lab.github.io/flock/

GNU Affero General Public License v3.0

288 stars 39 forks source link

feat(arena): optimize data frame deserialization #453

Closed gangliao closed 2 years ago

gangliao commented 2 years ago

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

By submitting this pull request

[x] I confirm that my contribution is made under the terms of the GNU Affero General Public License v3.0 license.
[x] I confirm that I've made a best effort attempt to update all relevant documentation.

gangliao commented 2 years ago

We don't need to deserialize the data packet when we get it for the aggregate function. Instead, when we receive the whole window data, we can deserialize all of them. The data partitions can then be deserialized in parallel.