issues with env variables for kubernetes deployment

[agaldemas]

Hello Benoit @bobeal, si c'est toi qui voit mon issue, nous nous connaissons (Alain Galdemas de Eridanis) Let's go to the subject, I found that the neo4j URI spring.neo4j.uri = bolt://localhost:7687, is in entity-service/src/main/resources/application-docker.properties file, like spring.kafka.bootstrap-servers, but those parameters are not exposed through environment variable. This is problematic and requires to keep entity-service in the same kubernetes namespace as neo4j !

there should be variables NEO4J_URI and SPRING_KAFKA_BOOTSTRAP_SERVERS to add like NEO4J_DEFAULT_DATABASE, NEO4J_PASSWORD.

We think about making a Helm chart for deployment, and want to have neo4j in a specific namespace, so that we need to add those variables, can you explain where and how we can do this, we are ready to help we'll propose a PR with our contribution on kubernetes deployment.

[bobeal]

Hello @agaldemas,

Indeed, these variables should be exposed in the k8s configuration.

It should be straightforward to add them since entity service will automatically bind (at least) what is defined in https://github.com/stellio-hub/stellio-context-broker/blob/develop/entity-service/src/main/resources/application.properties.

So for instance, if you add a SPRING_NEOJ_URI environment variable in the same way as the other environment variables (add an entry in the config map and use it in the deployment configuration), it will be handled automatically by the service.

PS : I've just merged a PR that integrates some recent updates on the k8s configuration (https://github.com/stellio-hub/stellio-context-broker/pull/653)

[agaldemas]

Hello @bobeal, OK clear, thanks a lot for this helpful hint. We'll try and shall propose a PR

CU soon !

[agaldemas]

Hello @bobeal ,

I managed to deploy successfully stellio, on our OVH dev cluster, but still with every components in the same namespace, this is a good news... But to success I had to remove your new variable KAFKA_LISTENERS, and some other things to have everything working, in particular the neo4j password substitution, is missing for the migration step, and you must keep the default password. I'll check again to confirm, and will open a specific issue if needed !

to separate timescale in a separate namespace will be easy, as well neo4j, but remains the problem of the unique configmap, shared by all components...

to workaroud this, we need to introduce another way to manage configuration, may be directly by using helm charts, which is our final goal, and much more simple to manage than deployment...

Voilà pour les news !

[agaldemas]

Hello I confirm the issue if you use another password than "stellio_password", stellio entity service doesn't start correctly :

we get the error :

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'neo4jMigrationsInitializer' defined in class path resource [ac/simons/neo4j/migrations/springframework/boot/autoconfigure/MigrationsAutoConfiguration.class]: Invocation of init method failed; nested exception is org.neo4j.driver.exceptions.AuthenticationException: The client is unauthorized due to authentication failure.
2022-05-24 08:23:29,300 [ main] INFO o.neo4j.driver.internal.InternalDriver - info - Closing driver instance 1395502325
2022-05-24 08:23:29,302 [ main] INFO o.n.d.i.async.pool.ConnectionPoolImpl - info - Closing connection pool towards neo4j:7687
2022-05-24 08:23:29,777 [ main] INFO ConditionEvaluationReportLoggingListener - logMessage -
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-05-24 08:23:29,873 [ main] ERROR o.s.boot.SpringApplication - reportFailure - Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'neo4jMigrationsInitializer' defined in class path resource [ac/simons/neo4j/migrations/springframework/boot/autoconfigure/MigrationsAutoConfiguration.class]: Invocation of init method failed; nested exception is org.neo4j.driver.exceptions.AuthenticationException: The client is unauthorized due to authentication failure.

if I put back "stellio_password", and recreate every components, the stellio-entity-service start & everything works fine !

[agaldemas]

the other thing I had to correct is to simplify the kafka configuration : only use :

       - name: KAFKA_ADVERTISED_LISTENERS
          value: EXTERNAL://stellio-kafka:9092

in deployment no need of KAFKA_LISTENERS, else it makes an error and Kafka don't start ! see my kafka-deployment.yaml :

apiVersion: v1
kind: Service
metadata:
  labels:
    app: stellio-kafka
  name: stellio-kafka
spec:
  ports:
    - name: "9092"
      port: 9092
      targetPort: 9092
  selector:
    app: stellio-kafka
status:
  loadBalancer: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: stellio-kafka
  name: stellio-kafka
spec:
  selector:
    matchLabels:
      app: stellio-kafka
  replicas: 1
  strategy: {}
  template:
    metadata:
      labels:
        app: stellio-kafka
    spec:
      containers:
      - env:
        # - name: MY_POD_IP
        #   valueFrom:
        #     fieldRef:
        #       fieldPath: status.podIP
        # - name: KAFKA_LISTENERS
        #   valueFrom:
        #     configMapKeyRef:
        #       name: stellio-conf
        #       key: KAFKA_LISTENERS
        - name: KAFKA_ADVERTISED_LISTENERS
          value: EXTERNAL://stellio-kafka:9092
        - name: KAFKA_BROKER_ID
          value: "1"
        - name: KAFKA_INTER_BROKER_LISTENER_NAME
          value: EXTERNAL
        - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
          value: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
        - name: KAFKA_LOG4J_ROOT_LOGLEVEL
          value: INFO
        - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
          value: "1"
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: zookeeper:2181
        image: confluentinc/cp-kafka:5.4.1
        name: stellio-kafka
        # ports:
        # - containerPort: 9092
        #   name: stellio-kafka
        resources: {}
      restartPolicy: Always
status: {}

for this I think I can work on fixing it, since it's outside the kotlin code ;O) !

[agaldemas]

if you want I close this issue and do a new one only for the neo4j password problem ?

[bobeal]

I think it can be closed. @agaldemas re-open it if it's not the case.