search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, MiniCPM, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, GraphRAG, DeepSpeed, vLLM, FastChat, Axolotl, etc.
Apache License 2.0
6.51k stars 1.24k forks source link

[PPML] the output file .csv.cbc of simplequery cannot be decrypted successfully #5220

Closed liu-shaojun closed 2 years ago

liu-shaojun commented 2 years ago

Description

when running the following script in ppml graphene client container, it outputs the encrypted file part-00000-3f0345e5-76ca-41b3-84e4-af4969d77725-c000.csv.cbc, this csv.cbc file cannot be decrypted with Decrypt.scala

bash bigdl-ppml-submit.sh \
        --master $RUNTIME_SPARK_MASTER \
        --deploy-mode client \
        --sgx-enabled true \
        --sgx-log-level error \
        --sgx-driver-memory 64g \
        --sgx-driver-jvm-memory 12g \
        --sgx-executor-memory 64g \
        --sgx-executor-jvm-memory 12g \
        --driver-memory 32g \
        --driver-cores 4 \
        --executor-memory 32g \
        --executor-cores 4 \
        --conf spark.kubernetes.container.image=$RUNTIME_K8S_SPARK_IMAGE \
        --num-executors 2 \
        --conf spark.cores.max=8 \
        --name simplequery \
        --verbose \
        --class com.intel.analytics.bigdl.ppml.examples.SimpleQuerySparkExample \
        --jars local://$SPARK_HOME/examples/jars/scopt_2.12-3.7.1.jar,$(echo $BIGDL_HOME/jars/* |tr ' ' ',' | sed "s#${BIGDL_HOME}#local://${BIGDL_HOME}#g") \
        local://$BIGDL_HOME/jars/bigdl-ppml-spark_3.1.2-2.1.0-SNAPSHOT.jar \
        --inputPath /ppml/trusted-big-data-ml/work/data/ppml_e2e_demo/data/ \
        --outputPath /ppml/trusted-big-data-ml/work/data/ppml_e2e_demo/data/encrypted \
        --inputEncryptModeValue plain_text \
        --outputEncryptModeValue AES/CBC/PKCS5Padding \
        --primaryKeyPath /ppml/trusted-big-data-ml/work/data/ppml_e2e_demo/key/ehsm_encrypted_primary_key \
        --dataKeyPath /ppml/trusted-big-data-ml/work/data/ppml_e2e_demo/key/ehsm_encrypted_data_key \
        --kmsType EHSMKeyManagementService \
        --kmsServerIP 172.168.0.242 \
        --kmsServerPort 9000 \
        --ehsmAPPID $appid \
        --ehsmAPPKEY $appkey

The error message is MicrosoftTeams-image (2)

However, using Encrypt.scala to encrypt people.csv it will get people.csv.encrypted file, then use Decrypt.scala to decrypt people.csv.encrypted, it can be decrypted successfully.

liu-shaojun commented 2 years ago

This issue no longer occurs with the latest bigdl-ppml-spark_3.1.2-2.1.0-SNAPSHOT.jar.