acrlabs / simkube

Record-and-replay Kubernetes simulator based on KWOK
https://appliedcomputing.io/simkube
MIT License
164 stars 7 forks source link

Replace with KWOK implementation #91

Closed drmorr0 closed 5 months ago

drmorr0 commented 5 months ago

Description

After testing, we've confirmed that SimKube works fine with KWOK and so we're ripping out all of the virtual-kubelet-based implementations. This only left skctl as the last remaining bit of Go code in the repo, and so for build tooling simplicity, this has been replaced with a Rust implementation.

Testing done

Additional info

Resolves #70, #71

drmorr0 commented 5 months ago

Kubernetes Object DAG

%%{init: {'themeVariables': {'mainBkg': '#ddd'}}}%%
graph LR

classDef default color:#000
subgraph global
  direction LR
  global/simkube[<b>Namespace</b><br>simkube]
%% DELETED OBJECTS START
%% DELETED OBJECTS END
end

subgraph sk-tracer
  direction LR
  simkube/sk-tracer-svc[<b>Service</b><br>sk-tracer-svc]
  simkube/sk-tracer-depl[<b>Deployment</b><br>sk-tracer-depl]
  simkube/sk-tracer-sa[<b>ServiceAccount</b><br>sk-tracer-sa]
  sk-tracer/sk-tracer-crb[<b>ClusterRoleBinding</b><br>sk-tracer-crb]
  simkube/sk-tracer-tracer-config[<b>ConfigMap</b><br>sk-tracer-tracer-config]
  simkube/sk-tracer-sa--->simkube/sk-tracer-depl
  sk-tracer/sk-tracer-crb--->simkube/sk-tracer-depl
  simkube/sk-tracer-tracer-config--->simkube/sk-tracer-depl
%% DELETED OBJECTS START
%% DELETED OBJECTS END
end

subgraph sk-ctrl
  direction LR
  simkube/sk-ctrl-depl[<b>Deployment</b><br>sk-ctrl-depl]
  simkube/sk-ctrl-sa[<b>ServiceAccount</b><br>sk-ctrl-sa]
  sk-ctrl/sk-ctrl-crb[<b>ClusterRoleBinding</b><br>sk-ctrl-crb]
  simkube/sk-ctrl-sa--->simkube/sk-ctrl-depl
  sk-ctrl/sk-ctrl-crb--->simkube/sk-ctrl-depl
%% DELETED OBJECTS START
%% DELETED OBJECTS END
end

subgraph test
  direction LR
  simkube/test-depl[<b>Deployment</b><br>test-depl]
%% DELETED OBJECTS START
%% DELETED OBJECTS END
end

global--->sk-tracer
global--->sk-ctrl
global--->test

%% STYLE DEFINITIONS START
  style kube-system/sk-cloudprov-svc fill:#e67
  style kube-system/sk-cloudprov-sa fill:#e67
  style sk-cloudprov/sk-cloudprov-crb fill:#e67
  style kube-system/sk-cloudprov-depl fill:#e67
  style kube-system/sk-cloudprov-cluster-autoscaler-sa fill:#e67
  style sk-cloudprov/sk-cloudprov-cluster-autoscaler-crb fill:#e67
  style kube-system/sk-cloudprov-cluster-autoscaler-config fill:#e67
  style kube-system/sk-cloudprov-cluster-autoscaler-depl fill:#e67
  style simkube/sk-vnode-sa fill:#e67
  style sk-vnode/sk-vnode-crb fill:#e67
  style simkube/sk-vnode-node-skeleton fill:#e67
  style simkube/sk-vnode-depl fill:#e67
  style simkube/test-depl fill:#cb4
%% STYLE DEFINITIONS END

New object Deleted object Updated object Updated object (causes pod recreation)

Detailed Diff

#### kube-system/sk-cloudprov-cluster-autoscaler-config: Removed ``` root: { "apiVersion": "v1", "kind": "ConfigMap", "metadata": { "name": "sk-cloudprov-cluster-autoscaler-config", "namespace": "kube-system" }, "data": { "config.yml": "---\naddress: sk-cloudprov-svc:8086\n" } } --> not present ```
#### kube-system/sk-cloudprov-cluster-autoscaler-depl: Removed ``` root: { "apiVersion": "apps/v1", "kind": "Deployment", "metadata": { "labels": { "fireconfig.io/app": "cluster-autoscaler" }, "name": "sk-cloudprov-cluster-autoscaler-depl", "namespace": "kube-system" }, "spec": { "replicas": 1, "selector": { "matchLabels": { "fireconfig.io/app": "cluster-autoscaler" } }, "template": { "metadata": { "labels": { "fireconfig.io/app": "cluster-autoscaler" } }, "spec": { "containers": [ { "args": [ "/cluster-autoscaler", "--cloud-provider", "externalgrpc", "--cloud-config", "/config/config.yml", "--scale-down-delay-after-add", "1m", "--scale-down-unneeded-time", "1m", "--v", "4" ], "env": [ { "name": "POD_OWNER", "value": "sk-cloudprov-cluster-autoscaler-depl" } ], "image": "localhost:5000/cluster-autoscaler:latest", "name": "cluster-autoscaler", "securityContext": { "capabilities": { "add": [ "SYS_PTRACE" ] } }, "volumeMounts": [ { "mountPath": "/config", "name": "cluster-autoscaler-config" } ] } ], "nodeSelector": { "type": "kind-control-plane" }, "serviceAccountName": "sk-cloudprov-cluster-autoscaler-sa", "tolerations": [ { "effect": "NoSchedule", "key": "node-role.kubernetes.io/control-plane", "value": "" } ], "volumes": [ { "configMap": { "items": [ { "key": "config.yml", "path": "config.yml" } ], "name": "sk-cloudprov-cluster-autoscaler-config" }, "name": "cluster-autoscaler-config" } ] } } } } --> not present ```
#### kube-system/sk-cloudprov-cluster-autoscaler-sa: Removed ``` root: { "apiVersion": "v1", "kind": "ServiceAccount", "metadata": { "name": "sk-cloudprov-cluster-autoscaler-sa", "namespace": "kube-system" } } --> not present ```
#### kube-system/sk-cloudprov-depl: Removed ``` root: { "apiVersion": "apps/v1", "kind": "Deployment", "metadata": { "labels": { "fireconfig.io/app": "sk-cloudprov" }, "name": "sk-cloudprov-depl", "namespace": "kube-system" }, "spec": { "replicas": 1, "selector": { "matchLabels": { "fireconfig.io/app": "sk-cloudprov" } }, "template": { "metadata": { "labels": { "fireconfig.io/app": "sk-cloudprov" } }, "spec": { "containers": [ { "args": [ "/sk-cloudprov", "-A", "fireconfig.io/app=sk-vnode" ], "env": [ { "name": "POD_OWNER", "value": "sk-cloudprov-depl" } ], "image": "PLACEHOLDER", "name": "sk-cloudprov", "ports": [ { "containerPort": 8086 } ], "securityContext": { "capabilities": { "add": [ "SYS_PTRACE" ] } } } ], "nodeSelector": { "type": "kind-worker" }, "serviceAccountName": "sk-cloudprov-sa" } } } } --> not present ```
#### kube-system/sk-cloudprov-sa: Removed ``` root: { "apiVersion": "v1", "kind": "ServiceAccount", "metadata": { "name": "sk-cloudprov-sa", "namespace": "kube-system" } } --> not present ```
#### kube-system/sk-cloudprov-svc: Removed ``` root: { "apiVersion": "v1", "kind": "Service", "metadata": { "name": "sk-cloudprov-svc", "namespace": "kube-system" }, "spec": { "ports": [ { "port": 8086, "targetPort": 8086 } ], "selector": { "fireconfig.io/app": "sk-cloudprov" } } } --> not present ```
#### simkube/sk-vnode-depl: Removed ``` root: { "apiVersion": "apps/v1", "kind": "Deployment", "metadata": { "labels": { "fireconfig.io/app": "sk-vnode" }, "name": "sk-vnode-depl", "namespace": "simkube" }, "spec": { "replicas": 1, "selector": { "matchLabels": { "fireconfig.io/app": "sk-vnode" } }, "template": { "metadata": { "labels": { "fireconfig.io/app": "sk-vnode" } }, "spec": { "containers": [ { "args": [ "/sk-vnode", "--node-skeleton", "/config/node.yml" ], "env": [ { "name": "POD_NAME", "valueFrom": { "fieldRef": { "fieldPath": "metadata.name" } } }, { "name": "POD_NAMESPACE", "valueFrom": { "fieldRef": { "fieldPath": "metadata.namespace" } } }, { "name": "POD_OWNER", "value": "sk-vnode-depl" } ], "image": "PLACEHOLDER", "name": "sk-vnode", "securityContext": { "capabilities": { "add": [ "SYS_PTRACE" ] } }, "volumeMounts": [ { "mountPath": "/config", "name": "node-skeleton" } ] } ], "nodeSelector": { "type": "kind-worker" }, "serviceAccountName": "sk-vnode-sa", "volumes": [ { "configMap": { "items": [ { "key": "node.yml", "path": "node.yml" } ], "name": "sk-vnode-node-skeleton" }, "name": "node-skeleton" } ] } } } } --> not present ```
#### simkube/sk-vnode-node-skeleton: Removed ``` root: { "apiVersion": "v1", "kind": "ConfigMap", "metadata": { "name": "sk-vnode-node-skeleton", "namespace": "simkube" }, "data": { "node.yml": "---\napiVersion: v1\nkind: Node\nstatus:\n allocatable:\n cpu: \"16\"\n memory: \"32Gi\"\n capacity:\n cpu: \"16\"\n memory: \"32Gi\"\n" } } --> not present ```
#### simkube/sk-vnode-sa: Removed ``` root: { "apiVersion": "v1", "kind": "ServiceAccount", "metadata": { "name": "sk-vnode-sa", "namespace": "simkube" } } --> not present ```
#### simkube/test-depl: ChangedWithPodRecreate ``` root['spec']['template']['spec']['tolerations'][0]['effect']: "NoExecute" --> "NoSchedule" ``` ``` root['spec']['template']['spec']['tolerations'][0]['key']: "simkube.io/virtual-node" --> "kwok-provider" ```
#### sk-cloudprov/sk-cloudprov-cluster-autoscaler-crb: Removed ``` root: { "apiVersion": "rbac.authorization.k8s.io/v1", "kind": "ClusterRoleBinding", "metadata": { "name": "sk-cloudprov-cluster-autoscaler-crb" }, "roleRef": { "apiGroup": "rbac.authorization.k8s.io", "kind": "ClusterRole", "name": "cluster-admin" }, "subjects": [ { "kind": "ServiceAccount", "name": "sk-cloudprov-cluster-autoscaler-sa", "namespace": "kube-system" } ] } --> not present ```
#### sk-cloudprov/sk-cloudprov-crb: Removed ``` root: { "apiVersion": "rbac.authorization.k8s.io/v1", "kind": "ClusterRoleBinding", "metadata": { "name": "sk-cloudprov-crb" }, "roleRef": { "apiGroup": "rbac.authorization.k8s.io", "kind": "ClusterRole", "name": "cluster-admin" }, "subjects": [ { "kind": "ServiceAccount", "name": "sk-cloudprov-sa", "namespace": "kube-system" } ] } --> not present ```
#### sk-vnode/sk-vnode-crb: Removed ``` root: { "apiVersion": "rbac.authorization.k8s.io/v1", "kind": "ClusterRoleBinding", "metadata": { "name": "sk-vnode-crb" }, "roleRef": { "apiGroup": "rbac.authorization.k8s.io", "kind": "ClusterRole", "name": "cluster-admin" }, "subjects": [ { "kind": "ServiceAccount", "name": "sk-vnode-sa", "namespace": "simkube" } ] } --> not present ```
codecov[bot] commented 5 months ago

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (ef25407) 58.60% compared to head (0c55e51) 72.44%.

Files Patch % Lines
src/time.rs 0.00% 3 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #91 +/- ## =========================================== + Coverage 58.60% 72.44% +13.84% =========================================== Files 38 23 -15 Lines 2215 1056 -1159 =========================================== - Hits 1298 765 -533 + Misses 893 291 -602 + Partials 24 0 -24 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.