search

intel / trusted-certificate-issuer

Trusted Certificate Service (TCS) is a K8s service to protect signing keys using Intel's SGX technology. K8s CSR and cert-manager CR APIs are both supported. TCS also contains integration samples for Istio service mesh and Key Management Reference Application (KMRA).
Apache License 2.0
29 stars 15 forks source link

Failed to open Intel SGX device #99

Open aseenaent opened 1 year ago

aseenaent commented 1 year ago

Trying to deploy tcs-issuer in k8s cluster, but got :

$ kubectl logs -f -n intel-system tci-tcs-issuer-5b8b5bf544-c55hv
Defaulted container "tcs-issuer" out of: tcs-issuer, init (init)
1.6934929015869703e+09  INFO    controller-runtime.metrics      Metrics server is starting to listen    {"addr": ":8082"}
[get_driver_type edmm_utility.cpp:116] Failed to open Intel SGX device.
[get_driver_type /home/sgx/jenkins/ubuntuServer2004-release-build-trunk-217/build_target/PROD/label/Builder-UbuntuSrv20/label_exp/ubuntu64/linux-trunk-opensource/psw/urts/linux/edmm_utility.cpp:116] Failed to open Intel SGX device.
1.69349290159065e+09    LEVEL(-2)       SGX     Failed to configure command
1.6934929015906732e+09  ERROR   setup   SGX initialization      {"error": "failed to initialize PKCS#11 library: pkcs11: 0x30: CKR_DEVICE_ERROR", "errorVerbose": "pkcs11: 0x30: CKR_DEVICE_ERROR\nfailed to initialize PKCS#11 library"}
main.main
        /workspace/main.go:102
runtime.main
        /go/src/runtime/proc.go:250
$ kubectl describe node mynode | grep sgx
                    feature.node.kubernetes.io/cpu-security.sgx.enabled=true
                    feature.node.kubernetes.io/cpu-sgx.enabled=true
                    intel.feature.node.kubernetes.io/sgx=true
                    nfd.node.kubernetes.io/extended-resources: sgx.intel.com/epc
  sgx.intel.com/enclave:    110
  sgx.intel.com/epc:        521138176
  sgx.intel.com/provision:  110
  sgx.intel.com/enclave:    110
  sgx.intel.com/epc:        521138176
  sgx.intel.com/provision:  110
  inteldeviceplugins-system   intel-sgx-plugin-cj84b                                    0 (0%)        0 (0%)      0 (0%)           0 (0%)         85m
  sgx.intel.com/enclave    1           1
  sgx.intel.com/epc        512Ki       512Ki
  sgx.intel.com/provision  0           0
$ sudo service aesmd status
● aesmd.service - Intel(R) Architectural Enclave Service Manager
     Loaded: loaded (/lib/systemd/system/aesmd.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2023-08-31 22:23:29 +08; 26min ago
   Main PID: 321422 (aesm_service)
      Tasks: 4 (limit: 629145)
     Memory: 5.0M
        CPU: 958ms
     CGroup: /system.slice/aesmd.service
             └─321422 /opt/intel/sgx-aesm-service/aesm/aesm_service

Ogos 31 22:23:28 p12sl01igoh groupadd[321085]: group added to /etc/group: name=sgx_prv, GID=1002
Ogos 31 22:23:28 p12sl01igoh groupadd[321085]: group added to /etc/gshadow: name=sgx_prv
Ogos 31 22:23:28 p12sl01igoh groupadd[321085]: new group: name=sgx_prv, GID=1002
Ogos 31 22:23:29 p12sl01igoh usermod[321337]: add 'aesmd' to group 'sgx_prv'
Ogos 31 22:23:29 p12sl01igoh usermod[321337]: add 'aesmd' to shadow group 'sgx_prv'
Ogos 31 22:23:29 p12sl01igoh usermod[321346]: add 'aesmd' to group 'sgx'
Ogos 31 22:23:29 p12sl01igoh usermod[321346]: add 'aesmd' to shadow group 'sgx'
Ogos 31 22:23:29 p12sl01igoh aesm_service[321398]: aesm_service: warning: Turn to daemon. Use "--no-daemon" option to execute in foreground.
Ogos 31 22:23:29 p12sl01igoh systemd[1]: Started Intel(R) Architectural Enclave Service Manager.
Ogos 31 22:23:29 p12sl01igoh aesm_service[321422]: The server sock is 0x560ede43d300
$ is-sgx-available
SGX supported by CPU: true
SGX1 (ECREATE, EENTER, ...): true
SGX2 (EAUG, EACCEPT, EMODPR, ...): true
Flexible Launch Control (IA32_SGXPUBKEYHASH{0..3} MSRs): true
SGX extensions for virtualizers (EINCVIRTCHILD, EDECVIRTCHILD, ESETCONTEXT): false
Extensions for concurrent memory management (ETRACKC, ELDBC, ELDUC, ERDINFO): false
CET enclave attributes support (See Table 37-5 in the SDM): false
Key separation and sharing (KSS) support (CONFIGID, CONFIGSVN, ISVEXTPRODID, ISVFAMILYID report fields): true
Max enclave size (32-bit): 0x80000000
Max enclave size (64-bit): 0x100000000000000
EPC size: 0x1f0ff000
SGX driver loaded: true
AESMD installed: true
SGX PSW/libsgx installed: true
#PF/#GP information in EXINFO in MISC region of SSA supported: true
#CP information in EXINFO in MISC region of SSA supported: false
$ kubectl get pods -A
NAMESPACE                   NAME                                                     READY   STATUS             RESTARTS        AGE
cert-manager                cert-manager-875c7579b-67dtq                             1/1     Running            1 (91m ago)     112m
kube-system                 coredns-77ccd57875-bfpn6                                 1/1     Running            1 (91m ago)     120m
kube-system                 local-path-provisioner-957fdf8bc-29szs                   1/1     Running            2 (91m ago)     120m
cert-manager                cert-manager-cainjector-7bb6786867-tjh9b                 1/1     Running            2 (91m ago)     112m
kube-system                 metrics-server-648b5df564-jml9r                          1/1     Running            2 (91m ago)     120m
inteldeviceplugins-system   inteldeviceplugins-controller-manager-68d4865b4b-b7pcl   2/2     Running            3 (91m ago)     104m
cert-manager                cert-manager-webhook-89dc55877-m2rh6                     1/1     Running            2 (91m ago)     112m
node-feature-discovery      node-feature-discovery-master-7f4b4cd8d9-fvh9w           1/1     Running            2 (91m ago)     111m
node-feature-discovery      node-feature-discovery-worker-st5dl                      1/1     Running            3 (91m ago)     111m
inteldeviceplugins-system   intel-sgx-plugin-cj84b                                   1/1     Running            0               88m
intel-system                tci-tcs-issuer-5b8b5bf544-c55hv                          0/1     CrashLoopBackOff   7 (4m47s ago)   15m
$ ll /dev/sgx*
crw-rw---- 1 root sgx     10, 125 Ogos 31 22:23 /dev/sgx_enclave
crw-rw---- 1 root sgx_prv 10, 126 Ogos 31 22:23 /dev/sgx_provision

/dev/sgx:
total 0
drwxr-xr-x  2 root root   80 Ogos 31 22:23 ./
drwxr-xr-x 20 root root 4680 Ogos 31 22:23 ../
lrwxrwxrwx  1 root root   14 Ogos 31 22:23 enclave -> ../sgx_enclave
lrwxrwxrwx  1 root root   16 Ogos 31 22:23 provision -> ../sgx_provision

Using in-tree SGX driver with kernel 6.2.0-26-generic.

Component Deploy With Version
cert-manager Helm 1.12.3
NFD Helm 0.13.3
Device Plugin Operator Helm 0.27.1
SGX Device Plugin Helm 0.27.1
TCS Helm 0.5.0
aseenaent commented 4 months ago

Bump