streamnative / pulsar-beat-output

Elastic Beats Output to Apache Pulsar
Apache License 2.0
54 stars 24 forks source link

[FEATURE] support : Multiple advertised listeners #97

Open Robustshouhualee opened 3 months ago

Robustshouhualee commented 3 months ago

backgroup

This tool currently does not support the configuration of :multiple converted listeners for pulsar. When the client's network environment is a combination of internal and external IPs, some network environments do not support writing to pulsar。

expect

Expect community authors to support this feature,thx。

Reference

https://pulsar.apache.org/docs/next/concepts-multiple-advertised-listeners/

Robustshouhualee commented 3 months ago

please look this issue。 @xuthus5 @tuteng @zhongxp1219

thetumbled commented 3 months ago

Could you help to support this? thanks. @tuteng @shoothzj

xuthus5 commented 3 months ago

Do you want to make the ClientOptions.ListenerName parameter support configurable? https://github.com/apache/pulsar-client-go/blob/2d51323103a65ba522dc73f56f052cc81d663245/pulsar/client.go#L133 @Robustshouhualee

thetumbled commented 3 months ago

Do you want to make the ClientOptions.ListenerName parameter support configurable? https://github.com/apache/pulsar-client-go/blob/2d51323103a65ba522dc73f56f052cc81d663245/pulsar/client.go#L133 @Robustshouhualee

It seems that the pulsar go client has support for configuring ListenerName, what should we do in this project if we want to configure it?

xuthus5 commented 3 months ago

Change the struct pulsarConfig and add a field such as:

ListenerName string `config:"listener_name"`

https://github.com/streamnative/pulsar-beat-output/blob/e180e577d8992cb883b20687f749afdb8cfb8bb9/pulsar/config.go#L41

change function initOptions add condition:

if config.ListenerName != "" {
    clientOptions.ListenerName = config.ListenerName
}

https://github.com/streamnative/pulsar-beat-output/blob/e180e577d8992cb883b20687f749afdb8cfb8bb9/pulsar/config.go#L129

then change xxx.yaml add listener_name = {listener_name } to test.