search

giros-dit / semantic-data-aggregator

A semantic monitoring framework for aggregating data from heterogeneous sources.
Apache License 2.0
5 stars 0 forks source link

Test TID Kubernetes #29

Open idomingu opened 2 years ago

idomingu commented 2 years ago

Description

TID already has a production Kubernetes cluster running on their premises. Once #23 is functional, we must validate the execution of Flink Application-based jobs in this cluster.

Dependencies

cristinapmz commented 2 years ago

Request access to Kubernetes cluster

I have requested access to Alejandro (the TID colleague who deployed the K8s cluster). He has created a namespace called sda to which he will give us access using a kubeconfig file. In this file, he has also created a single user called upm for the moment, but you can create as many users as you want.

cristinapmz commented 2 years ago

Document access to Kubernetes cluster

The corresponding kubeconfig file shall be used to access the TID K8s cluster. Specifically, the following option must be passed with the kubectl command.

kubectl --kubeconfig <name_file_kubeconfig> …

In the ... you specify the action you want to perform, e.g. get pods, get services, apply -f name_file_yaml, etc.

cristinapmz commented 2 years ago

There are several issues related to how the cluster has been deployed that are worth mentioning:

cristinapmz commented 2 years ago

Physical resources of the TID K8S cluster

The physical resources of each node in the TID K8s cluster are as follows:

daniel-gonzalez-sanchez commented 2 years ago

Request access to Kubernetes cluster

I have requested access to Alejandro (the TID colleague who deployed the K8s cluster). He has created a namespace called sda to which he will give us access using a kubeconfig file. In this file, he has also created a single user called upm for the moment, but you can create as many users as you want.

In order to install some requirements needed for the installation of the flink-on-k8s-operator, we needed management permissions outside the sda namespace. Therefore, the user upm has been granted full administration permissions. For any general change that affects the k8s cluster, it will be notified in advance.

daniel-gonzalez-sanchez commented 2 years ago

There are several issues related to how the cluster has been deployed that are worth mentioning:

  • The cluster access service via dashboard is currently not available. Alejandro has told me that it does not work properly when the credentials are passed to it, so he is trying to fix it and when it is ready, he will let us know.
  • We don't have permissions to list the cluster nodes. However, I attach the image he gave me where these nodes are listed (just in case this information could be relevant in the future). cluster-nodes
  • The helm tool has been installed in the control panel. However, it is not necessary that it is installed on the server but where it is going to be used (i.e. on the PC where kubectl is running).
  • A load balancer has not been configured for the time being as there was no activity in the cluster. Alejandro has told me that he is planning to implement it and that as soon as he does, he will let us know.

An additional requirement to include in the list would be the installation of a service in the k8s cluster for a private registry of Docker images. This would allow us to load custom Docker images.

cristinapmz commented 2 years ago

Private registry of Docker images

In the case of the TID cluster, the insecure private registry is available on the endpoint accessible via kubernetes:31320. The name kubernetes has been associated with the k8s cluster IP address which is 192.168.159.9. The latter must be indicated in the file /etc/hosts.

daniel-gonzalez-sanchez commented 2 years ago

Load Balancer

A Load Balancer based on Metallb and Nginx has been installed on the k8s cluster.