search

clusterpedia-io / clusterpedia

The Encyclopedia of Kubernetes clusters
https://clusterpedia.io
Apache License 2.0
791 stars 122 forks source link

support for dividing table for internal storage #647

Open calvin0327 opened 8 months ago

calvin0327 commented 8 months ago

What type of PR is this? /kind feature

What this PR does / why we need it:

Which issue(s) this PR fixes: Fixes https://github.com/clusterpedia-io/clusterpedia/issues/601

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

support for dividing table for internal storage
clusterpedia-bot commented 8 months ago

Hi @calvin0327, Thanks for your pull request! If the PR is ready, use the /auto-cc command to assign Reviewer to Review. We will review it shortly.

Details Instructions for interacting with me using comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the [gh-ci-bot](https://github.com/clusterpedia-io/gh-ci-bot) repository.
calvin0327 commented 8 months ago

I'll test the results later.

calvin0327 commented 8 months ago

The test results:

  1. Create pediacluster cr:
apiVersion: cluster.clusterpedia.io/v1alpha2
kind: PediaCluster
metadata:
  name: cluster-example
spec:
  apiserver: "https://10.30.43.43:6443"
  caData: 
  tokenData:
  certData: 
  keyData: 
  syncResources:
    - group: apps
      resources:
        - deployments
    - group: ""
      resources:
        - pods
  1. After runing clustersynchro-manager component and generate these tables in clusterpedia database:
image
calvin0327 commented 8 months ago

@Iceber The pr basic feature have been completed, can we have reveiw first?

calvin0327 commented 6 months ago

/retest

calvin0327 commented 6 months ago

/retest

calvin0327 commented 6 months ago

here are test results:

apiVersion: cluster.clusterpedia.io/v1alpha2
kind: PediaCluster
metadata:
  name: cluster-example
spec:
  apiserver: "https://10.30.43.43:6443"
  caData:
  tokenData:
  certData:
  keyData:
  syncResources:
    - group: apps
      resources:
        - deployments
    - group: ""
      resources:
        - pods

[root@master01 ~]# kubectl  get pediacluster
NAME              READY   VERSION   APISERVER
cluster-example   True    v1.29.2   https://10.6.212.13:6443
[root@master01 ~]#

# DivisionPolicy is "None"

mysql> show tables;
+------------------------+
| Tables_in_clusterpedia |
+------------------------+
| resources              |
+------------------------+
1 row in set (0.01 sec)
mysql>

[root@master01 ~]# kubectl --cluster clusterpedia get deploy -A
NAMESPACE             CLUSTER           NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
clusterpedia-system   cluster-example   clusterpedia-apiserver                1/1     1            1           3h17m
clusterpedia-system   cluster-example   clusterpedia-clustersynchro-manager   1/1     1            1           3h17m
clusterpedia-system   cluster-example   clusterpedia-controller-manager       1/1     1            1           3h17m
kube-system           cluster-example   cilium-operator                       2/2     2            2           77d
kube-system           cluster-example   coredns                               2/2     2            2           84d
kubeflow              cluster-example   training-operator                     1/1     1            1           24d
volcano-system        cluster-example   volcano-admission                     1/1     1            1           30d
volcano-system        cluster-example   volcano-controllers                   1/1     1            1           30d
volcano-system        cluster-example   volcano-scheduler                     1/1     1            1           30d

[root@master01 ~]# kubectl delete pediacluster cluster-example
pediacluster.cluster.clusterpedia.io "cluster-example" deleted

[root@master01 ~]# kubectl --cluster clusterpedia get deploy -A
Error from server (NotFound): Unable to list "apps/v1, Resource=deployments": the server could not find the requested resource (get deployments.apps)

# DivisionPolicy is "GVR"

mysql> show tables;
+------------------------+
| Tables_in_clusterpedia |
+------------------------+
| apps_v1_deployments    |
| v1_pods                |
+------------------------+
2 rows in set (0.00 sec)

mysql>

[root@master01 clusterpedia]# kubectl --cluster clusterpedia get deploy -A
NAMESPACE             CLUSTER           NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
clusterpedia-system   cluster-example   clusterpedia-apiserver                1/1     1            1           3h57m
clusterpedia-system   cluster-example   clusterpedia-clustersynchro-manager   1/1     1            1           3h57m
clusterpedia-system   cluster-example   clusterpedia-controller-manager       1/1     1            1           3h57m
kube-system           cluster-example   cilium-operator                       2/2     2            2           77d
kube-system           cluster-example   coredns                               2/2     2            2           84d
kubeflow              cluster-example   training-operator                     1/1     1            1           24d
volcano-system        cluster-example   volcano-admission                     1/1     1            1           30d
volcano-system        cluster-example   volcano-controllers                   1/1     1            1           30d
volcano-system        cluster-example   volcano-scheduler                     1/1     1            1           30d

[root@master01 clusterpedia]# kubectl delete pediacluster cluster-example
pediacluster.cluster.clusterpedia.io "cluster-example" deleted

[root@master01 clusterpedia]# kubectl --cluster clusterpedia get deploy -A
Error from server (NotFound): Unable to list "apps/v1, Resource=deployments": the server could not find the requested resource (get deployments.apps)