Coherence host ip in cpherence kafka sink connector

[myrulezzz]

Hi i am experimenting the coherence kafka sink connector. In the connector properties what is the parameter to define the coherence host ip?

[mgamanho]

It depends on what you want to do...

By default, you will have a full-blown Coherence member, storage-enabled or not (though it is recommended to separate functions, so non-storage-enabled is advised). In this case, there is no specific configuration necessary, you can just keep starting separate standalone Coherence members and they will form a cluster. Or you can change the defaults by including config files in the sink zip. These config files will be regular Coherence configuration bits.

Alternatively, Coherence will start basic services in the sink connector task, which allows you to connect via Extend. In this case you will provide a Coherence configuration for Extend just like you would any Extend client. For more information, see here. For this you will have to disable starting Coherence in server mode with the "coherence.server" property set to true. For more info, see here.

Let me know if you need more details.

[myrulezzz]

Hi i have one more question. I have a coherence app that is running with some cached configured.Is it possible to create another coherence with kafka libraries and copy any cache data from the source coherence to the target in order to consume it to kafka? Is there any other solution to this?

[mgamanho]

Hi,

there are a number of ways of doing it.

The simplest is probably to configure a second app connected to the first one through extend.

You could also add the kafka libraries/config to your existing app and perform a rolling restart (bounce 1 member at a time), but that assumes that there would be no change to existing caches, preferably. If you want to preserve the origin cache and have a copy, use the first method. If you want a single cluster, try the second.

There may be other ways, and keep in mind that the kafka enabled cluster must be CE 21.06 and above.

Let me know if you have questions, thanks.

[myrulezzz]

@mgamanho thanks i will have a look and let tou know.i need to figure out the proper way as coherence is running in kubernetes cluster

[mgamanho]

Sure, don't hesitate to reach out when necessary.

If using k8s, you may want to check out the coherence operator which adds a ton of useful features.

[myrulezzz]

@mgamanho in order to configure existing coherence cluster to send data to a new coherence node loaded with the kafka java classes do you have to edit existing coherence configuration? or the configuration is done on the new coherence node? I am looking for the possibility to copy data from Oracle ECE coherence to an external coherence loaded with the kafka classes without touching code of the Oracle ECE coherence if is possible

[mgamanho]

This is probably best handled in conjunction with the ECE team. And it will probably be faster.

We have a good working relationship with them, and if you like we can get in touch and find out the best approach?

Let me know and we'll take it from there. Thanks

[myrulezzz]

@mgamanho Yes of course. Any help is welcome. My experiment is to get ece data to kafka.

[myrulezzz]

Hi @mgamanho. Do you have any update on this?

[mgamanho]

Hi, working on it, will keep you posted ASAP.

[myrulezzz]

Thanks

[mgamanho]

Hi, consulted with the ECE team, and they feel that for technical, privacy and security reasons it would be best to accomplish this through official support channels. As an ECE customer you should be able to open a ticket. Feel free to keep me involved through that ticket. Thanks