Closed vi0eros closed 2 months ago
Hi @vi0eros
PPML Image is designed for SGX enabled platform
(recommended platform is 3rd-4th Xeon platform with SGX enabled on BIOS). If you can finish running the example with apache/spark:v3.1.3
image, while get errors with PPML image. Please check if SGX is enabled on your platform and then if SGX is set up with enough EPC (SGX reserved memory).
BTW, Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
is just a warning. It will keep show this message until job gets enough resource.
Hi @vi0eros PPML Image is designed for
SGX enabled platform
(recommended platform is 3rd-4th Xeon platform with SGX enabled on BIOS). If you can finish running the example withapache/spark:v3.1.3
image, while get errors with PPML image. Please check if SGX is enabled on your platform and then if SGX is set up with enough EPC (SGX reserved memory).BTW,
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
is just a warning. It will keep show this message until job gets enough resource.
@qiyuangong
Thank you for your response. I have verified that I am indeed running on an SGX-enabled platform. However, my EPC memory is somewhat limited. To address this, I have set the following parameter in start-pyspark-pi-on-k8s-client-sgx.sh
:
--conf spark.kubernetes.sgx.enabled=false
Specifically, I am wondering if the PPML image can run without SGX enabled, given that my EPC memory is quite limited.
I have also removed the resources section from spark-executor-template.yaml
. The updated content of spark-executor-template.yaml
is as follows:
apiVersion: v1
kind: Pod
spec:
containers:
- name: spark-executor
env:
- name: ATTESTATION
value: false
- name: ATTESTATION_URL
value: your_attestation_url
- name: MALLOC_ARENA_MAX
value: 4
volumeMounts:
- name: device-plugin
mountPath: /var/lib/kubelet/device-plugins
- name: aesm-socket
mountPath: /var/run/aesmd/aesm.socket
- name: nfs-storage
mountPath: /ppml/data
# Removed resources section
volumes:
- name: device-plugin
hostPath:
path: /var/lib/kubelet/device-plugins
- name: aesm-socket
hostPath:
path: /var/run/aesmd/aesm.socket
- name: nfs-storage
persistentVolumeClaim:
claimName: nfsvolumeclaim
Is it possible to run the PPML image in a non-SGX mode, and if so, are there any additional configurations or adjustments needed to ensure that it functions correctly under these conditions?
Thank you for your assistance.
Hi @vi0eros PPML Image is designed for
SGX enabled platform
(recommended platform is 3rd-4th Xeon platform with SGX enabled on BIOS). If you can finish running the example withapache/spark:v3.1.3
image, while get errors with PPML image. Please check if SGX is enabled on your platform and then if SGX is set up with enough EPC (SGX reserved memory). BTW,Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
is just a warning. It will keep show this message until job gets enough resource.@qiyuangong
Thank you for your response. I have verified that I am indeed running on an SGX-enabled platform. However, my EPC memory is somewhat limited. To address this, I have set the following parameter in
start-pyspark-pi-on-k8s-client-sgx.sh
:--conf spark.kubernetes.sgx.enabled=false
Specifically, I am wondering if the PPML image can run without SGX enabled, given that my EPC memory is quite limited.
I have also removed the resources section from
spark-executor-template.yaml
. The updated content ofspark-executor-template.yaml
is as follows:apiVersion: v1 kind: Pod spec: containers: - name: spark-executor env: - name: ATTESTATION value: false - name: ATTESTATION_URL value: your_attestation_url - name: MALLOC_ARENA_MAX value: 4 volumeMounts: - name: device-plugin mountPath: /var/lib/kubelet/device-plugins - name: aesm-socket mountPath: /var/run/aesmd/aesm.socket - name: nfs-storage mountPath: /ppml/data # Removed resources section volumes: - name: device-plugin hostPath: path: /var/lib/kubelet/device-plugins - name: aesm-socket hostPath: path: /var/run/aesmd/aesm.socket - name: nfs-storage persistentVolumeClaim: claimName: nfsvolumeclaim
Is it possible to run the PPML image in a non-SGX mode, and if so, are there any additional configurations or adjustments needed to ensure that it functions correctly under these conditions?
Thank you for your assistance.
Hi @vi0eros BigDL PPML is mainly designed for TEE (SGX is one of them). Without the protection of TEE, we cannot ensure the confidentiality and integrity of the main components.
The example and image you chose are designed to run Apache Spark within Intel SGX. spark.kubernetes.sgx.enabled=false
is just for debugging, not for production. If this example is not running in SGX, we cannot ensure spark components are fully secured. In that case, changing to an Apache Spark image may be a better choice.
Given this information, I have decided to switch to using an Apache Spark image for my needs. This resolves my issue. Thank you for your assistance.
Given this information, I have decided to switch to using an Apache Spark image for my needs. This resolves my issue. Thank you for your assistance.
You are welcome. :)
Description:
When executing the script
ppml/trusted-bigdata/scripts/start-pyspark-pi-on-k8s-client-sgx.sh
using the$RUNTIME_K8S_SPARK_IMAGE
, I encountered a resource allocation error. The error log shows the following message repeatedly:However, when using the
apache/spark:v3.1.3
image, the job runs successfully and completes as expected.Script Details:
The script ppml/trusted-bigdata/scripts/start-pyspark-pi-on-k8s-client-sgx.sh is as follows:
Logs with PPML Image:
Script with Apache Spark Image:
Logs with Apache Spark Image:
Additional Information:
Building the PPML Image:
The PPML image is built using the following script
ppml/trusted-bigdata/custom-image/build-custom-image.sh
:Starting the PPML Image:
The PPML image is started using the following commands:
I am a beginner, please help me.