search

lightbend / cloudflow

Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.
https://cloudflow.io
Apache License 2.0
321 stars 89 forks source link

"Address already in use" for one of two AkkaServerStreamlets from separate modules in runLocal #960

Open unit7-0 opened 3 years ago

unit7-0 commented 3 years ago

Describe the bug When defining two streamlets extending AkkaServerStreamlet in separate pipeline modules, the same port(3000) is selected for both streamlets in runLocal mode. Accordingly, only one streamlet is running, which managed to occupy the free port. For another one, an error message appears in the logs: [ERROR] [17:46:47.871] ak.io.TcpListener:159 Bind failed for TCP channel on endpoint [/0.0.0.0:3000] java.net.BindException: [/0.0.0.0:3000] Address already in use

To Reproduce Define two streamlets of the base type AkkaServerStreamlet in separate pipeline modules and try to run in local mode.

Expected behavior Both streamlets are running and bound to different ports.

Additional logs:

---------------------------- Streamlets per project ----------------------------
ingestor-2 - output file: file:/tmp/cloudflow-local-run3676335750737767568/ingestor-2-local.log

    cdr-ingress2 [carly.ingestor.CallRecordIngress2]
    - HTTP port [3000]

spark-aggregation - output file: file:/tmp/cloudflow-local-run3676335750737767568/spark-aggregation-local.log

    cdr-aggregator [carly.aggregator.CallStatsAggregator]
    cdr-generator1 [carly.aggregator.CallRecordGeneratorIngress]
    cdr-generator2 [carly.aggregator.CallRecordGeneratorIngress]

akka-java-aggregation-output - output file: file:/tmp/cloudflow-local-run3676335750737767568/akka-java-aggregation-output-local.log

    console-egress [carly.output.AggregateRecordEgress]
    error-egress [carly.output.InvalidRecordEgress]

akka-cdr-ingestor - output file: file:/tmp/cloudflow-local-run3676335750737767568/akka-cdr-ingestor-local.log

    cdr-ingress [carly.ingestor.CallRecordIngress]
    - HTTP port [3000]
    split [carly.ingestor.CallRecordSplit]

I suppose this is due to the fact that each module runs in a separate JVM, but the ports for server streamlets are selected in the local scope of the module relative to the base port offset: https://github.com/lightbend/cloudflow/blob/master/core/cloudflow-localrunner/src/main/scala/cloudflow/localrunner/LocalRunner.scala#L137

andreaTP commented 3 years ago

Hi @unit7-0 and thanks for this report, sorry for the long time waiting. What you are saying has been "by-design" introduced here by @RayRoestenburg : https://github.com/lightbend/cloudflow/commit/55f022f3c96363712e59b3dfced54cc585020f78

I will double-check with him on what's the expected behavior. I see, anyhow, that we have most likely a bug here: https://github.com/lightbend/cloudflow/commit/55f022f3c96363712e59b3dfced54cc585020f78#diff-89d991b0c4bfbbd6d221723b70a341bba15274296e893417d710358da8296fd9R128 and here: https://github.com/lightbend/cloudflow/commit/55f022f3c96363712e59b3dfced54cc585020f78#diff-01d3a7c1e14d9d9f5cc6d61e91b7b89ea86087750afcf1f8690295de8e7efc2dR235

where endpointIdx get's incremented and not used afterward.