Deployment with Kubernetes: typhonAnalyticsConfig

[MarieSaphira]

Hi @zolotas4 and @patrickneubauer , the analytics component uses this to connect to the containers. Instead of entering the values for K8S_EXTERNAL_IP:NODE_PORT by hand, the DL component (that knows about their values) could also generate this file. What's your opinion?

[zolotas4]

Hi @MarieSaphira. Correct it would be good to have this file generated by TyphonDL. for both Docker and Kubernetes deployments.

[MarieSaphira]

Hi, could you explain what exactly should be in that file? This is what I would generate so far: Three general topics:

• pre event topic
• post event topic
• authorization event topic

What about the use-case specific topics? Should they be added? Or does the user have to do that themselves?

In your example, you also have connection info to the databases. This is only possible if the analytics component is in the same network. So

# -- Maria DB Thanos PC
database.url=jdbc:mariadb://localhost:3308/Alpha
database.username=thanos
database.password=

from here, would be

# -- Maria DB Polystore
database.url=alphadb:3606
database.username=thanos
database.password=

So does the "loading events from a database" section also have to be generated?

[zolotas4]

Hi @MarieSaphira, only lines 1-22 of this should be generated.

The rest (e.g., other kafka topics, maria db connections, sql binlog csv loading) are user-defined configurations we added for different simulators we created and also for scenario specific storage of results. But none of these are part of what it should be generated - users will need to add them if needed.

[MarieSaphira]

Ok, but in your kubernetes deployment, zookeeper is not published. But you need it to be published @patrickneubauer ?

[patrickneubauer]

Let's stick with the recommendations of Strimzi and not publish Zookeeper services. This will also avoid increasing configuration complexity.

In regards to K8S_EXTERNAL_IP:NODE_PORT, these values are established either before or after TyphonDL generates the Kubernetes YAML file and depends on the existence of a running Kubernetes pool. In other words, the IP and node port may be available before or after. I suggest to leave this open as well as it will provide the opportunity to deploy the configuration to different instances and avoid the regeneration of the configuration for (only) this purpose. Does that make sense?

In case it needs to be known at generation-time, please add this step to the TyphonDL user guide. Thanks!