search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc
Apache License 2.0
6.74k stars 1.27k forks source link

[PPML] the output file .csv.cbc of simplequery cannot be decrypted successfully #5220

Closed liu-shaojun closed 2 years ago

liu-shaojun commented 2 years ago

Description

when running the following script in ppml graphene client container, it outputs the encrypted file part-00000-3f0345e5-76ca-41b3-84e4-af4969d77725-c000.csv.cbc, this csv.cbc file cannot be decrypted with Decrypt.scala

bash bigdl-ppml-submit.sh \
        --master $RUNTIME_SPARK_MASTER \
        --deploy-mode client \
        --sgx-enabled true \
        --sgx-log-level error \
        --sgx-driver-memory 64g \
        --sgx-driver-jvm-memory 12g \
        --sgx-executor-memory 64g \
        --sgx-executor-jvm-memory 12g \
        --driver-memory 32g \
        --driver-cores 4 \
        --executor-memory 32g \
        --executor-cores 4 \
        --conf spark.kubernetes.container.image=$RUNTIME_K8S_SPARK_IMAGE \
        --num-executors 2 \
        --conf spark.cores.max=8 \
        --name simplequery \
        --verbose \
        --class com.intel.analytics.bigdl.ppml.examples.SimpleQuerySparkExample \
        --jars local://$SPARK_HOME/examples/jars/scopt_2.12-3.7.1.jar,$(echo $BIGDL_HOME/jars/* |tr ' ' ',' | sed "s#${BIGDL_HOME}#local://${BIGDL_HOME}#g") \
        local://$BIGDL_HOME/jars/bigdl-ppml-spark_3.1.2-2.1.0-SNAPSHOT.jar \
        --inputPath /ppml/trusted-big-data-ml/work/data/ppml_e2e_demo/data/ \
        --outputPath /ppml/trusted-big-data-ml/work/data/ppml_e2e_demo/data/encrypted \
        --inputEncryptModeValue plain_text \
        --outputEncryptModeValue AES/CBC/PKCS5Padding \
        --primaryKeyPath /ppml/trusted-big-data-ml/work/data/ppml_e2e_demo/key/ehsm_encrypted_primary_key \
        --dataKeyPath /ppml/trusted-big-data-ml/work/data/ppml_e2e_demo/key/ehsm_encrypted_data_key \
        --kmsType EHSMKeyManagementService \
        --kmsServerIP 172.168.0.242 \
        --kmsServerPort 9000 \
        --ehsmAPPID $appid \
        --ehsmAPPKEY $appkey

The error message is MicrosoftTeams-image (2)

However, using Encrypt.scala to encrypt people.csv it will get people.csv.encrypted file, then use Decrypt.scala to decrypt people.csv.encrypted, it can be decrypted successfully.

liu-shaojun commented 2 years ago

This issue no longer occurs with the latest bigdl-ppml-spark_3.1.2-2.1.0-SNAPSHOT.jar.