search

numaproj / numaflow-java

Numaflow Java SDK
Apache License 2.0
22 stars 10 forks source link

chore: initial code #1

Closed dseapy closed 1 year ago

dseapy commented 1 year ago

Signed-off-by: David Seapy dseapy@gmail.com

sarabala1979 commented 1 year ago

@dseapy sorry for the delay, we are in kubecon. I will review it this weekend.

dseapy commented 1 year ago

@dseapy sorry for the delay, we are in kubecon. I will review it this weekend.

No rush at all, enjoy kubecon!

sarabala1979 commented 1 year ago

@dseapy how is concurrency on the gRPC server it is similar to the HTTP server? Do we need to handle in function or sink level?

dseapy commented 1 year ago

@dseapy how is concurrency on the gRPC server it is similar to the HTTP server? Do we need to handle in function or sink level?

Concurrency is handled by grpc-java, specifically the executor passed to the serverbuilder. Currently this PR doesn't override the default executor used by the grpc-java serverbuilder, so a cached thread pool is used. There is a lot of discussion about which executor is optimal (ref: https://github.com/grpc/grpc-java/issues/7381), for which there seems to be varying opinions/trade-offs.

dseapy commented 1 year ago

@dseapy how is concurrency on the gRPC server it is similar to the HTTP server? Do we need to handle in function or sink level?

Concurrency is handled by grpc-java, specifically the executor passed to the serverbuilder. Currently this PR doesn't override the default executor used by the grpc-java serverbuilder, so a cached thread pool is used. There is a lot of discussion about which executor is optimal (ref: grpc/grpc-java#7381), for which there seems to be varying opinions/trade-offs.

There are also FunctionServer and SinkServer constructors (i.e. https://github.com/numaproj/numaflow-java/pull/1/files#diff-b46ed048cbc73098c1fe762914bbfba6e1558503a24752f4c3c990647da1687fR45) for overriding the default executor by providing a custom ServerBuilder if someone using the sdk wants to pass in their own.