search

NVIDIA / k8s-device-plugin

NVIDIA device plugin for Kubernetes
Apache License 2.0
2.45k stars 573 forks source link

Enhance e2e logging #677

Closed ArangoGutierrez closed 3 weeks ago

ArangoGutierrez commented 3 weeks ago

Now it looks like this

❯ make -f tests/e2e/Makefile e2e-test
go test -v /Users/eduardoa/src/github/nvidia/k8s-device-plugin/tests/e2e -args \
                -kubeconfig=/Users/eduardoa/.kube/config \
                -driver-enabled=true \
                -image.repo=nvcr.io/nvidia/k8s-device-plugin \
                -image.tag=v0.15.0 \
                -image.pull-policy=IfNotPresent \
                -log-artifacts=/Users/eduardoa/src/github/nvidia/k8s-device-plugin/e2e_logs \
                -helm-chart=/Users/eduardoa/src/github/nvidia/k8s-device-plugin/deployments/helm/nvidia-device-plugin \
                -ginkgo.focus="\[nvidia\]" \
                -test.timeout=1h \
                -ginkgo.v
=== RUN   TestE2E
Running Suite: nvidia k8s-device-plugin e2e suite - /Users/eduardoa/src/github/nvidia/k8s-device-plugin/tests/e2e
=================================================================================================================
Random Seed: 1713895500 - will randomize all specs

Will run 6 of 6 specs
------------------------------
[nvidia] GPU Device Plugin When deploying k8s-device-plugin and NV Driver is installed it should create nvidia.com/gpu resource
/Users/eduardoa/src/github/nvidia/k8s-device-plugin/tests/e2e/device-plugin_test.go:141
  STEP: Creating a kubernetes client @ 04/23/24 20:05:00.931
  STEP: Building a namespace with basename k8s-device-plugin @ 04/23/24 20:05:00.933
  STEP: Creating a helm client @ 04/23/24 20:05:01.752
  STEP: Installing k8s-device-plugin Helm chart @ 04/23/24 20:05:01.761
  STEP: Checking the node capacity @ 04/23/24 20:05:14.123
  STEP: [Cleanup] Deleting testing namespace "k8s-device-plugin-9162". @ 04/23/24 20:05:20.381
• [19.746 seconds]
------------------------------
[nvidia] GPU Device Plugin When deploying k8s-device-plugin and NV Driver is installed it should run GPU jobs
/Users/eduardoa/src/github/nvidia/k8s-device-plugin/tests/e2e/device-plugin_test.go:160
  STEP: Creating a kubernetes client @ 04/23/24 20:05:20.676
  STEP: Building a namespace with basename k8s-device-plugin @ 04/23/24 20:05:20.678
  STEP: Creating a helm client @ 04/23/24 20:05:20.992
  STEP: Installing k8s-device-plugin Helm chart @ 04/23/24 20:05:20.999
  STEP: Creating a GPU-job @ 04/23/24 20:05:33.028
  STEP: Waiting for job to complete @ 04/23/24 20:05:33.282
  STEP: [Cleanup] Deleting testing namespace "k8s-device-plugin-7868". @ 04/23/24 20:05:49.624
• [29.230 seconds]
------------------------------
[nvidia] GPU Feature Discovery When deploying GFD and NV Driver is not installed it should create nvidia.com timestamp label
/Users/eduardoa/src/github/nvidia/k8s-device-plugin/tests/e2e/gpu-feature-discovery_test.go:173
  STEP: Creating a kubernetes client @ 04/23/24 20:05:49.909
  STEP: Building a namespace with basename gpu-feature-discovery @ 04/23/24 20:05:49.911
  STEP: Creating a helm client @ 04/23/24 20:05:50.202
  STEP: Installing GFD Helm chart @ 04/23/24 20:05:50.203
  STEP: Checking the node labels @ 04/23/24 20:06:25.036
  STEP: [Cleanup] Deleting NFD labels, annotations and taints from node holodeck @ 04/23/24 20:06:38.529
  STEP: [Cleanup] Deleting testing namespace "gpu-feature-discovery-3473". @ 04/23/24 20:06:39.436
• [49.795 seconds]
------------------------------
[nvidia] GPU Feature Discovery When deploying GFD and NV Driver is not installed and the NodeFeature API is enabled gfd should create node feature object
/Users/eduardoa/src/github/nvidia/k8s-device-plugin/tests/e2e/gpu-feature-discovery_test.go:193
  STEP: Creating a kubernetes client @ 04/23/24 20:06:39.703
  STEP: Building a namespace with basename gpu-feature-discovery @ 04/23/24 20:06:39.704
  STEP: Creating a helm client @ 04/23/24 20:06:39.933
  STEP: Installing GFD Helm chart @ 04/23/24 20:06:39.935
  STEP: Updating GFD Helm chart values @ 04/23/24 20:07:19.761
  STEP: Checking if NodeFeature CR object is created @ 04/23/24 20:08:35.121
  STEP: Checking that node labels are created from NodeFeature object @ 04/23/24 20:08:35.888
  STEP: [Cleanup] Deleting NFD labels, annotations and taints from node holodeck @ 04/23/24 20:08:53.009
  STEP: [Cleanup] Deleting NodeFeature objects from namespace gpu-feature-discovery-7794 @ 04/23/24 20:08:53.82
  STEP: [Cleanup] Deleting testing namespace "gpu-feature-discovery-7794". @ 04/23/24 20:08:54.058
• [134.592 seconds]
------------------------------
[nvidia] GPU Feature Discovery When deploying GFD and NV Driver is installed it should create nvidia.com labels
/Users/eduardoa/src/github/nvidia/k8s-device-plugin/tests/e2e/gpu-feature-discovery_test.go:229
  STEP: Creating a kubernetes client @ 04/23/24 20:08:54.295
  STEP: Building a namespace with basename gpu-feature-discovery @ 04/23/24 20:08:54.296
  STEP: Creating a helm client @ 04/23/24 20:08:54.539
  STEP: Installing GFD Helm chart @ 04/23/24 20:08:54.544
  STEP: Checking the node labels @ 04/23/24 20:09:35.926
  STEP: [Cleanup] Deleting NFD labels, annotations and taints from node holodeck @ 04/23/24 20:09:59.164
  STEP: [Cleanup] Deleting testing namespace "gpu-feature-discovery-3122". @ 04/23/24 20:10:00.061
• [66.155 seconds]
------------------------------
[nvidia] GPU Feature Discovery When deploying GFD and NV Driver is installed and the NodeFeature API is enabled gfd should create node feature object
/Users/eduardoa/src/github/nvidia/k8s-device-plugin/tests/e2e/gpu-feature-discovery_test.go:247
  STEP: Creating a kubernetes client @ 04/23/24 20:10:00.449
  STEP: Building a namespace with basename gpu-feature-discovery @ 04/23/24 20:10:00.451
  STEP: Creating a helm client @ 04/23/24 20:10:00.845
  STEP: Installing GFD Helm chart @ 04/23/24 20:10:00.847
  STEP: Updating GFD Helm chart values @ 04/23/24 20:10:42.519
  STEP: Checking if NodeFeature CR object is created @ 04/23/24 20:11:59.333
  STEP: Checking that node labels are created from NodeFeature CR object @ 04/23/24 20:12:00.033
  STEP: [Cleanup] Deleting NFD labels, annotations and taints from node holodeck @ 04/23/24 20:12:21.568
  STEP: [Cleanup] Deleting NodeFeature objects from namespace gpu-feature-discovery-2242 @ 04/23/24 20:12:22.557
  STEP: [Cleanup] Deleting testing namespace "gpu-feature-discovery-2242". @ 04/23/24 20:12:22.818
• [142.770 seconds]
------------------------------

Ran 6 of 6 Specs in 442.292 seconds
SUCCESS! -- 6 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestE2E (442.29s)
PASS
ok      github.com/NVIDIA/k8s-device-plugin/tests/e2e   443.061s