Infinispan Chart for cross site mode

[ic2hrmk]

Hello,

I am looking for an opportunity to deploy the InfiniSpan in a cross-DC mode.

Does this chart support this mode? And, if yes, are there any documentation pages available?

Thanks!

[ryanemerson]

The chart doesn't support xsite.

It should be possible for xsite to work, however it would require the StatefulSet template to be modified to expose the required ports and additional Services to be created. More information on the configuration required can be found in our xsite guide.

I recommend that you consider using the Infinispan Operator for your k8s deployment. The operator provides xsite support OOB and is the simplest way to get started.

[ic2hrmk]

Hello @ryanemerson, thank you for the clarification!

[fallen-up]

@ryanemerson maybe you can show configuration template for jgroup and xsite in yml format? I'm having trouble finding the right config :(

changing helm-chart is not scary, I will add the necessary ports.

[ryanemerson]

@fallen-up Our intention is to eventually add a dedicated helm chart for xsite deployments. @pruivo Do we have an ETA for this?

[fallen-up]

@ryanemerson that's great, but it's definitely gonna take some time. I would be very grateful for an example config :)

Doesn't work:

deploy:
  infinispan:
    jgroups:
      stack:
        relay:
          xmlns: "urn:org:jgroups"
          site: NL
          max_site_masters: "1000"
        remote-sites:
          remote-site:
            - name: NL
            - name: LD
          default-stack: tcp
        name: xsite
        extends: udp
    cacheContainer:
      name: default
      transport:
        cluster: ${infinispan.cluster.name:cluster}
        node-name: ${infinispan.node.name:}
        stack: xsite

[pruivo]

No ETA since the interest in this feature is low (it is the first time I heard someone request it this year) and it is not trivial to implement. I would suggest to use the operator for this (https://infinispan.org/docs/infinispan-operator/main/operator.html#setting-up-xsite).

The main problem to solve is to make the pods accessible between clusters. The operator does that by deploying a gossip router and making it accessible to both clusters. A new JGroups stack must be created to use the TUNNEL protocol (http://jgroups.org/manual5/index.html#TUNNEL_Advanced) .

[fallen-up]

@pruivo it's not big problem for me to link services between different k8s-clusters. but I need help with correct config for jgroup in yml-format.

Could you provide correct format? Or maybe some lifehack with external files? for example:

deploy:
  infinispan:
    jgroups:
      stack-file:
        name: tcp
        path: /opt/jgroups-tcp.xml

[pruivo]

we don't need to link services, pods IPs should be accessible from both clusters. If that is the case, you need a headless service which includes the IPs of all pods in both cluster and and configure DNS_PING as in https://github.com/infinispan/infinispan/blob/main/core/src/main/resources/default-configs/default-jgroups-kubernetes.xml#L28