As technology, I should support Clusterizable independent unit per technical or functional concern

[olivierlemee]

https://www.notion.so/cybnity/447-d01de61153714443ae8fc294300b773a

REQ_MAIN4: https://www.notion.so/cybnity/REQ_MAIN_4-8513483dd519412087185e24134453bc?pvs=4 As Clusterizable independent unit per technical or functional concern:

• One Vert.x Verticle is deployable per JVM and per CPU core for each technical concern (e.g Bridge for routing a an API http/rest or event bus API exposed to Internet) according to the exposed protocol, and with securized isolation (e.g dedicated settings and sla)
• Each technical service (e.g Vert.x bridge Verticle for protocol routing as backend gateway) or functional service (e.g UI capability entry point exposing a capabilities domain’s api; one feature Verticle per JVM per CPU core on computation units) is packaged/deployable as an independent applicative component on an independent JVM and CPU core (e.g allowing horizontal scalability).

Components: vertx server, reactjs, docker, K8s cluster

TODO

• [x] create helm github repository supporting the helm chart source codes

  • [x] configure repositoy, workflows...
  • [ ] branch rules, roles, permission

• [x] develop multi-environments support from charts/sub-charts with migration of existing sh scripts (Node areas)

  • see help docs
    • https://github.com/marketplace/actions/helm-deploy
    • https://medium.com/swlh/deploying-to-kubernetes-with-helm-and-github-actions-14825e6df1f2
    • https://helm.sh/docs/howto/chart_releaser_action/

• [x] develop workflow CI (based on existing helm chart for one envt without specific cluster configuration) that deploy a version into a github environnement (not self-hosted used with public repo; minikube based) allowing to test components during a common development, or during a tests campaign execution of Test stage

• [x] Infrastructure services area

  • [x] Defense platform cluster as Kubernetes Local Dev cluster (deployable as mono instance cluster of modules into a Minikube stack)

    • [x] Provisioning management of networking, load balancer, database, users, permissions... and Kubernetes cluster (orchestration management) via Helm (Local cluster for dev on workstation's Minikube; "Staging & QA" cluster for test on OVHCloud)

    • [x] Docker image & resources configuration management via Helm source codes (Configuration of systems' resources, logs, forwarding ports of Docker image, Kubernetes objects (e.g baseline/live replica cluster allowing pre-prod dress rehearsal, canary cluster for code changed and rollout with automated recovery, production/live cluster) supporting environments/infrastructures deployment AT APPLICATION LEVEL

    • [ ] Control Plane

    • [ ] API server (including https public root certificates) and Controller Manager > Cloud provider API link > cluster external ip address

    • [ ] Volumes configuration as stateful set (and \deployment\local-deployment.yaml file definition)

    • [x] User interfaces area as Kubernetes Node (including external ip address)

    • [ ] Kube-Proxy (including inside/outside cluster network session rules)

    • [x] Domains I/O area Node (including external ip address)

    • [ ] Kube-Proxy (including inside/outside cluster network session rules)

    • [x] Domains area Node (including external ip address)

    • [ ] Kube-Proxy (including inside/outside cluster network session rules)

    • [x] Infrastructure Services Area Node (including external ip address)

    • [ ] Kube-Proxy (including inside/outside cluster network session rules)

  • [ ] Infrastructure services zone firewall
  • [ ] Domains zone firewall as Calico enforcer, kubernetes network policy

• [x] Foundation common modules

  • [ ] UI area
    • [ ] Internet DMZ firewall as calico enforcer, kubernetes network policy
    • [x] UIS brokers registry system as zookeeper cluster (helm project)
      • [x] Kubernetes control plan (\service\uis-broker-service.yaml file definition)
      • [x] Kubernetes control plan (\service\uis-broker-systems-deployment.yaml file definition)
      • [x] Volumes configuration as stateful set linking to systems-deployment file
      • [x] Kubernetes POD (see Assembly strategy )
      • [x] Kubernetes Service (including POD VPN address, POD unique IP address)
    • [x] Users Interactions Space - UIS system as Redis cluster (helm project)
      • [x] Kubernetes control plan (\service\uis-service.yaml file definition)
      • [x] Kubernetes control plan (\service\uis-systems-deployment.yaml file definition)
      • [x] Volumes configuration as stateful set linking to systems-deployment file
      • [x] Kubernetes POD (see Assembly strategy )
      • [x] Kubernetes Service (including POD VPN address, POD unique IP address)
    • [x] Reactive messaging gateway system as vert.x backend server (helm project)
      • [x] Kubernetes control plan (\service\backend-service.yaml file definition)
      • [x] Kubernetes control plan (\service\backend-systems-deployment.yaml file definition)
      • [x] Volumes configuration as stateful set linking to systems-deployment file
      • [x] Kubernetes POD (see Assembly strategy )
      • [x] Kubernetes Service (including POD VPN address, POD unique IP address)
    • [x] Web reactive frontend system as ReactJS and Node.js server (helm project)
      • [x] Kubernetes control plan (\service\frontend-service.yaml file definition)
      • [x] Kubernetes control plan (\service\frontend-systems-deployment.yaml file definition)
      • [x] Volumes configuration as stateful set linking to systems-deployment file
      • [x] Kubernetes POD (see Assembly strategy )
      • [x] Kubernetes Service (including POD VPN address, POD unique IP address)
  • [x] Domains I/O area
    • [ ] Domains API zone firewall as calico enforcer, kubernetes network policy
    • [x] DIS brokers registry as Zookeeper cluster (helm project)
      • [x] Kubernetes control plan (\service\dis-broker-service.yaml file definition)
      • [x] Kubernetes control plan (\service\dis-broker-systems-deployment.yaml file definition)
      • [x] Volumes configuration as stateful set linking to systems-deployment file
      • [x] Kubernetes POD (see Assembly strategy )
      • [x] Kubernetes Service (including POD VPN address, POD unique IP address)
    • [x] Domains Interactions Space as Kafka cluster (helm project)
      • [x] Kubernetes control plan (\service\dis-service.yaml file definition)
      • [x] Kubernetes control plan (\service\dis-systems-deployment.yaml file definition)
      • [x] Volumes configuration as stateful set linking to systems-deployment file
      • [x] Kubernetes POD (see Assembly strategy )
      • [x] Kubernetes Service (including POD VPN address, POD unique IP address)
  • [x] Domains area

• [ ] Access control domain deployable modules

  • [x] Infrastructure services area
  • [ ] UI area
    • [ ] Keycloack SSO backoffice & database server as PostgreSQL server, Keycloack backoffice server supporting the API delegation for teams members permissions, accounts etc.. managed by a final user (e.g CISO) : Helm project
      • [x] Keycloack SSO service (defined in infrastructure area over Kubernetes Node) and in UI layer (only login view module) as delegation login screen from CYBNITY frontend
      • [ ] delegation adapter usable from access control domain gateway to check sso from many areas (ui for sso login, doamin for sso token check)
    • [ ] access control ui - e.g security team member permissions, ui capabilities server
      • [ ] Kubernetes control plan (\service\access-control-domain-ui-service.yaml file definition)
      • [ ] Kubernetes control plan (\service\access-control-domain-ui-systems-deployment.yaml file definition)
      • [ ] Volumes configuration as stateful set linking to systems-deployment file
      • [ ] Kubernetes POD (see Assembly strategy )
      • [ ] Kubernetes Service (including POD VPN address, POD unique IP address)
  • [ ] Domains I/O area
    • [ ] access control domain gateway system as Vert.x server
      • [ ] Kubernetes control plan (\service\access-control-domain-gw-service.yaml file definition)
      • [ ] Kubernetes control plan (\service\access-control-domain-gateway-systems-deployment.yaml file definition)
      • [ ] Volumes configuration as stateful set linking to systems-deployment file
      • [ ] Kubernetes POD (see Assembly strategy )
      • [ ] Kubernetes Service (including POD VPN address, POD unique IP address)
  • [ ] Domains area
    • [ ] Migrate Postgresql module from ui layer into the infrastructure layer (including access to keycloack back office) + open network connection between SSO service to node
    • [ ] Access control process module system as Flink server (rts process module, apache flink, apache flink cep, kafka client, mongodb client, zookeeper client)
      • [ ] Flink server (Docker image) including port as dedicated Docker image template
      • [ ] Kubernetes control plan (\service\access-control-domain-process-service.yaml file definition)
      • [ ] Kubernetes control plan (\service\access-control-domain-process-systems-deployment.yaml file definition)
      • [ ] Volumes configuration as stateful set linking to systems-deployment file
      • [ ] Kubernetes POD (see Assembly strategy )
      • [ ] Kubernetes Service (including POD VPN address, POD unique IP address)

• [ ] Update the Implementation View documentation regarding:

  • [ ] the packaging models (docker and kubernetes components level)
  • [ ] the configuration-management (helm resources and settings level per specific docker image)
  • [ ] the system IaC (terraform IaC and systems level)
  • [ ] update the containerized system documentation about systems platform
  • [ ] the runtime dependencies overview between deployable components and macro-links between them (e.g CYBNITY system components level between park of CYBNITY Docker Images executed into Kubernetes services activated)
  • [ ] the map of logical stream ports (e.g internal / external) helping to understand the path of events/interactions flow between the Kubernetes Nodes (supposed deployable as application Space Based Architecture; with visibility on location where security policy about permissions/flow opened are controlling the runtime perimeters)

• tech: kubernetes, docker, terraform, helm

[olivierlemee]

• [x] migrate systems charts source code from Foundation to new project
  • [x] start with isolated Node definition via helm
    • See help docs
    • https://kubernetes.io/docs/concepts/architecture/nodes/
    • https://www.baeldung.com/ops/kubernetes-helm
    • https://codefresh.io/blog/helm-deployment-enrivonments/
    • multiple envts https://blog.tarkalabs.com/handling-multiple-environments-with-helm-kubernetes-f214192f8f7b
    • minikube use: https://minikube.sigs.k8s.io/docs/tutorials/setup_minikube_in_github_actions/
    • github using envts: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment , https://docs.github.com/en/actions/deployment/about-deployments/deploying-with-github-actions
• [x] develop CI-CD workflow allowing to build Helm charts, validate them, package to version, push into Dockerhub repository according to LOCAL DEV, QA environments
  • https://github.com/helm/chart-releaser

[olivierlemee]

Knowledge repository (janusgraph, cassandra) helm project implemented and included into the cluster IAS project