secretflow / heu

A high-performance homomorphic encryption algorithm library.
https://www.secretflow.org.cn/docs/heu/en/
Apache License 2.0
90 stars 41 forks source link

[Bug]: The code works well on the x86 Ubuntu server but fails on the ARM Ubuntu server. #147

Closed yongboy closed 3 months ago

yongboy commented 4 months ago

Issue Type

Usability

HEU Version

sf-heu 0.5.1b0

OS Platform and Distribution

Arm Ubuntu 20.04

Python Version

3.9

Compiler Version

No response

Current Behavior?

I generated the public/private key pair on an x86 Ubuntu 20.04 server and used the public key on an aarch64 (Arm Cortex-A55) Ubuntu 20.04 server.

The code below works well on the x86 Ubuntu server but fails on the ARM server.

The traceback:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/user/miniconda3/envs/live/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/home/user/miniconda3/envs/live/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/my_app/app.py", line 65, in activation
    slowfast_pred = encryptor.encrypt(heu_arr)
heu.phe.PheRuntimeError: [Enforce fail at heu/library/algorithms/paillier_float/internal/codec.cc:22] scalar.CompareAbs(pk_.PlaintextBound()) <= 0. integer scalar should in +/- 0, but get A7B46
Stacktrace:
#0 heu::lib::algorithms::paillier_f::internal::Codec::Encode()+0x7f82877e6c
#1 heu::lib::algorithms::paillier_f::Encryptor::Encrypt()+0x7f82875f24
#2 heu::lib::numpy::DoCallEncrypt<>()::{lambda()#1}::operator()()+0x7f825882c4
#3 std::_Function_handler<>::_M_invoke()+0x7f8286a1e4
#4 std::__future_base::_State_baseV2::_M_do_set()+0x7f82869ccc
#5 __pthread_once_slow+0x7fbc16b3b8
#6 std::_Function_handler<>::_M_invoke()+0x7f8286a654
#7 yacl::ThreadPool::WorkLoop()+0x7f8286ca40
#8 execute_native_thread_routine+0x7f8297d5b0
#9 start_thread+0x7fbc162624

The code:

from heu import numpy as hnp
from heu import phe

......

with open("public.key", "rb") as f:
    pk_buffer = f.read()

kit = hnp.setup(pickle.loads(pk_buffer))
evaluator = kit.evaluator()
encryptor = kit.encryptor()
encoder = phe.FloatEncoder(phe.SchemaType.ZPaillier)

......

                slowfast_pred = slowfast.predict(frames, bboxes)
                slowfast_pred = slowfast_pred.cpu()
                slowfast_pred = slowfast_pred.detach().numpy()

                heu_arr = hnp.array(slowfast_pred, encoder)
                slowfast_pred = encryptor.encrypt(heu_arr) # the line 65
usafchn commented 3 months ago

The reason is that the enum value numbers of phe.SchemaType.xxxx are misaligned, causing the algorithm to not be invoked correctly at the lower level.

I think your environment might involve one machine being x86 and the other being ARM, and the number of schemas supported by x86 and ARM are different, leading to the sequence numbers of this SchemaType enum being different. If both sides are either ARM or x86, there should be no issue.

We will fix this problem shortly.

usafchn commented 3 months ago

Please recompile and reinstall the code from the main branch on both sides.

yongboy commented 3 months ago

Thank you very much, it’s working perfectly!

I compiled the latest main branch code of this project on servers with different CPU architectures into the following files and updated them on several service instances one by one.

  1. sf_heu-0.6.0.dev0-cp310-cp310-manylinux_2_28_aarch64.whl
  2. sf_heu-0.6.0.dev0-cp310-cp310-manylinux2014_x86_64.whl

The installation command is as follows:

pip install sf_heu-0.6.0.dev0-cp310-cp310-manylinux_2_28_aarch64.whl --force-reinstall

Then:

  1. Delete the existing public and private keys
  2. Generate new public and private keys
  3. Distribute the new public and private keys to all aarch64 and x86_64 server instances
  4. All instances are working correctly

PS:

Code for generating the public and private keys:

from heu import phe

ckit = phe.setup(phe.SchemaType.ZPaillier, 2048)

with open('./public.key', 'wb') as f:
    f.write(ckit.public_key().serialize())

with open('./secret.key', 'wb') as f:
    f.write(ckit.secret_key().serialize())

print("Done!")

usafchn commented 3 months ago

Aha! Great! I'm glad to hear this news.