ibarrond / Pyfhel

PYthon For Homomorphic Encryption Libraries, perform encrypted computations such as sum, mult, scalar product or matrix multiplication in Python, with NumPy compatibility. Uses SEAL/PALISADE as backends, implemented using Cython.
https://pyfhel.readthedocs.io/
Apache License 2.0
487 stars 78 forks source link

An installation problem faced on Linux #203

Closed Kuuuuubrick closed 1 year ago

Kuuuuubrick commented 1 year ago

Description Hi there, I have faced some installation problem on my linux server

Code To Reproduce Error First of all, I have followed the prompt you provided to install with the command git clone --recursive https://github.com/ibarrond/Pyfhel.git, everything went well

git clone --recursive https://github.com/ibarrond/Pyfhel.git
Cloning into 'Pyfhel'...
remote: Enumerating objects: 3492, done.
remote: Counting objects: 100% (1682/1682), done.
remote: Compressing objects: 100% (434/434), done.
remote: Total 3492 (delta 1316), reused 1483 (delta 1222), pack-reused 1810
Receiving objects: 100% (3492/3492), 15.35 MiB | 5.99 MiB/s, done.
Resolving deltas: 100% (2289/2289), done.
Submodule 'SEAL' (https://github.com/microsoft/SEAL) registered for path 'Pyfhel/backend/SEAL'
Submodule 'palisade' (https://gitlab.com/palisade/palisade-release) registered for path 'Pyfhel/backend/palisade'
Cloning into '/root/PyCrCNN/Pyfhel/Pyfhel/backend/SEAL'...
remote: Enumerating objects: 17111, done.        
remote: Counting objects: 100% (282/282), done.        
remote: Compressing objects: 100% (152/152), done.        
remote: Total 17111 (delta 141), reused 228 (delta 111), pack-reused 16829        
Receiving objects: 100% (17111/17111), 5.01 MiB | 8.76 MiB/s, done.
Resolving deltas: 100% (12944/12944), done.
Cloning into '/root/PyCrCNN/Pyfhel/Pyfhel/backend/palisade'...
warning: redirecting to https://gitlab.com/palisade/palisade-release.git/
remote: Enumerating objects: 3776, done.        
remote: Counting objects: 100% (1568/1568), done.        
remote: Compressing objects: 100% (712/712), done.        
remote: Total 3776 (delta 919), reused 1422 (delta 840), pack-reused 2208        
Receiving objects: 100% (3776/3776), 11.94 MiB | 8.71 MiB/s, done.
Resolving deltas: 100% (2239/2239), done.
Submodule path 'Pyfhel/backend/SEAL': checked out '82b07db635132e297282649e2ab5908999089ad2'
Submodule path 'Pyfhel/backend/palisade': checked out '19e84ae53ecdd1ca3360c4cdbb6df2eb31f21195'
Submodule 'third-party/cereal' (https://gitlab.com/palisade/cereal.git) registered for path 'Pyfhel/backend/palisade/third-party/cereal'
Submodule 'third-party/google-benchmark' (https://github.com/google/benchmark.git) registered for path 'Pyfhel/backend/palisade/third-party/google-benchmark'
Submodule 'third-party/google-test' (https://github.com/google/googletest.git) registered for path 'Pyfhel/backend/palisade/third-party/google-test'
Submodule 'third-party/gperftools' (https://github.com/gperftools/gperftools.git) registered for path 'Pyfhel/backend/palisade/third-party/gperftools'
Cloning into '/root/PyCrCNN/Pyfhel/Pyfhel/backend/palisade/third-party/cereal'...
remote: Enumerating objects: 16500, done.        
remote: Counting objects: 100% (409/409), done.        
remote: Compressing objects: 100% (238/238), done.        
remote: Total 16500 (delta 216), reused 237 (delta 132), pack-reused 16091        
Receiving objects: 100% (16500/16500), 14.00 MiB | 6.75 MiB/s, done.
Resolving deltas: 100% (12395/12395), done.
Cloning into '/root/PyCrCNN/Pyfhel/Pyfhel/backend/palisade/third-party/google-benchmark'...
remote: Enumerating objects: 7965, done.        
remote: Counting objects: 100% (294/294), done.        
remote: Compressing objects: 100% (194/194), done.        
remote: Total 7965 (delta 142), reused 195 (delta 82), pack-reused 7671        
Receiving objects: 100% (7965/7965), 2.57 MiB | 4.42 MiB/s, done.
Resolving deltas: 100% (5267/5267), done.
Cloning into '/root/PyCrCNN/Pyfhel/Pyfhel/backend/palisade/third-party/google-test'...
remote: Enumerating objects: 26485, done.        
remote: Counting objects: 100% (62/62), done.        
remote: Compressing objects: 100% (49/49), done.        
remote: Total 26485 (delta 24), reused 31 (delta 12), pack-reused 26423        
Receiving objects: 100% (26485/26485), 12.27 MiB | 7.77 MiB/s, done.
Resolving deltas: 100% (19659/19659), done.
Cloning into '/root/PyCrCNN/Pyfhel/Pyfhel/backend/palisade/third-party/gperftools'...
remote: Enumerating objects: 7019, done.        
remote: Counting objects: 100% (150/150), done.        
remote: Compressing objects: 100% (88/88), done.        
remote: Total 7019 (delta 83), reused 97 (delta 62), pack-reused 6869        
Receiving objects: 100% (7019/7019), 3.92 MiB | 8.11 MiB/s, done.
Resolving deltas: 100% (5566/5566), done.
Submodule path 'Pyfhel/backend/palisade/third-party/cereal': checked out 'a384b101e4437b5a83b91c9f4e4fd2eae1d5c9e8'
Submodule path 'Pyfhel/backend/palisade/third-party/google-benchmark': checked out 'e451e50e9b8af453f076dec10bd6890847f1624e'
Submodule path 'Pyfhel/backend/palisade/third-party/google-test': checked out '8b4817e3df3746a20502a84580f661ac448821be'
Submodule path 'Pyfhel/backend/palisade/third-party/gperftools': checked out 'c1d546d7b22cc503f37e9a6efa1d249be60243a3'

And I'm pretty sure that I have installed every submodules.

After that, I have modified the file pyproject.toml to set the SEAL_THROW_ON_TRANSPARENT_CIPHERTEXT to OFF, then I tried to install with the command pip install .

cd Pyfhel/
pip install .
Processing /root/PyCrCNN/Pyfhel
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [39 lines of output]

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
                      other = np.repeat(other, nslots)
                  if (other.ndim==1) and \
                      (np.issubdtype(other.dtype, np.number)):
                      if self.scheme == Scheme_t.bfv:
                          return self._pyfhel.encodeInt(other.astype(np.int64)[:nslots])
                      elig self.scheme == Scheme_t.bgv:
                           ^
      ------------------------------------------------------------

      Pyfhel/PyCtxt.pyx:789:21: Syntax error in simple statement list
      Compiling Pyfhel/Pyfhel.pyx because it changed.
      Compiling Pyfhel/PyCtxt.pyx because it changed.
      Compiling Pyfhel/PyPtxt.pyx because it changed.
      Compiling Pyfhel/PyPoly.pyx because it changed.
      [1/4] Cythonizing Pyfhel/PyCtxt.pyx
      Traceback (most recent call last):
        File "/root/anaconda3/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
          main()
        File "/root/anaconda3/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/root/anaconda3/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-sw49npbn/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 177, in get_requires_for_build_wheel
          return self._get_build_requires(
        File "/tmp/pip-build-env-sw49npbn/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 159, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-sw49npbn/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 174, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 660, in <module>
          ext_modules=cythonize(
        File "/tmp/pip-build-env-sw49npbn/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
          cythonize_one(*args)
        File "/tmp/pip-build-env-sw49npbn/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: Pyfhel/PyCtxt.pyx
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

In order to obtain more information, I tried to use the command pip install -v ., here is the result

pip install -v .
Using pip 23.1.2 from /root/anaconda3/envs/pycrcnn/lib/python3.10/site-packages/pip (python 3.10)
Processing /root/PyCrCNN/Pyfhel
  Running command pip subprocess to install build dependencies
  Collecting setuptools<=60.9
    Using cached setuptools-60.9.0-py3-none-any.whl (1.1 MB)
  Collecting wheel
    Using cached wheel-0.40.0-py3-none-any.whl (64 kB)
  Collecting cython==3.0.0b1
    Using cached Cython-3.0.0b1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.5 MB)
  Collecting numpy>=1.21
    Using cached numpy-1.25.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.6 MB)
  Collecting cmake>=3.15
    Using cached cmake-3.26.4-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (24.0 MB)
  Collecting toml>=0.10
    Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
  Installing collected packages: cmake, wheel, toml, setuptools, numpy, cython
  Successfully installed cmake-3.26.4 cython-3.0.0b1 numpy-1.25.0 setuptools-60.9.0 toml-0.10.2 wheel-0.40.0
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  Installing build dependencies ... done
  Running command Getting requirements to build wheel

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
                  other = np.repeat(other, nslots)
              if (other.ndim==1) and \
                  (np.issubdtype(other.dtype, np.number)):
                  if self.scheme == Scheme_t.bfv:
                      return self._pyfhel.encodeInt(other.astype(np.int64)[:nslots])
                  elig self.scheme == Scheme_t.bgv:
                       ^
  ------------------------------------------------------------

  Pyfhel/PyCtxt.pyx:789:21: Syntax error in simple statement list
  Compiling Pyfhel/Pyfhel.pyx because it changed.
  Compiling Pyfhel/PyCtxt.pyx because it changed.
  Compiling Pyfhel/PyPtxt.pyx because it changed.
  Compiling Pyfhel/PyPoly.pyx because it changed.
  [1/4] Cythonizing Pyfhel/PyCtxt.pyx
  Traceback (most recent call last):
    File "/root/anaconda3/envs/pycrcnn/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/root/anaconda3/envs/pycrcnn/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/root/anaconda3/envs/pycrcnn/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-9fn9gp4k/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 177, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "/tmp/pip-build-env-9fn9gp4k/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 159, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-9fn9gp4k/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 174, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 660, in <module>
      ext_modules=cythonize(
    File "/tmp/pip-build-env-9fn9gp4k/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
      cythonize_one(*args)
    File "/tmp/pip-build-env-9fn9gp4k/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
      raise CompileError(None, pyx_file)
  Cython.Compiler.Errors.CompileError: Pyfhel/PyCtxt.pyx
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /root/anaconda3/envs/pycrcnn/bin/python /root/anaconda3/envs/pycrcnn/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpu2bwscwe
  cwd: /root/PyCrCNN/Pyfhel
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Setup:

Kuuuuubrick commented 1 year ago

Description After that, I tried to install on my local Windows PC for more info

Code To Reproduce Error

Likewise, I used git clone --recursive https://github.com/ibarrond/Pyfhel.git to clone the whole project with submodules.

After that, I have modified the file pyproject.toml to set the SEAL_THROW_ON_TRANSPARENT_CIPHERTEXT to OFF, then I tried to install with the command pip install -v .

(pycrcnn) D:\Project\Python\PyCrCNN\Pyfhel>pip install . -v
Using pip 23.1.2 from D:\Environment\Anaconda\envs\pycrcnn\lib\site-packages\pip (python 3.10)
Processing d:\project\python\pycrcnn\pyfhel
  Running command pip subprocess to install build dependencies
  Collecting setuptools<=60.9
    Using cached setuptools-60.9.0-py3-none-any.whl (1.1 MB)
  Collecting wheel
    Using cached wheel-0.40.0-py3-none-any.whl (64 kB)
  Collecting cython==3.0.0b1
    Using cached Cython-3.0.0b1-cp310-cp310-win_amd64.whl (2.7 MB)
  Collecting numpy>=1.21
    Using cached numpy-1.25.0-cp310-cp310-win_amd64.whl (15.0 MB)
  Collecting cmake>=3.15
    Using cached cmake-3.26.4-py2.py3-none-win_amd64.whl (33.0 MB)
  Collecting toml>=0.10
    Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
  Installing collected packages: cmake, wheel, toml, setuptools, numpy, cython
  Successfully installed cmake-3.26.4 cython-3.0.0b1 numpy-1.25.0 setuptools-60.9.0 toml-0.10.2 wheel-0.40.0
  Installing build dependencies ... done
  Running command Getting requirements to build wheel

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
                  other = np.repeat(other, nslots)
              if (other.ndim==1) and \
                  (np.issubdtype(other.dtype, np.number)):
                  if self.scheme == Scheme_t.bfv:
                      return self._pyfhel.encodeInt(other.astype(np.int64)[:nslots])
                  elig self.scheme == Scheme_t.bgv:
                       ^
  ------------------------------------------------------------

  Pyfhel\PyCtxt.pyx:789:21: Syntax error in simple statement list
  Compiling Pyfhel\Pyfhel.pyx because it changed.
  Compiling Pyfhel\PyCtxt.pyx because it changed.
  Compiling Pyfhel\PyPtxt.pyx because it changed.
  Compiling Pyfhel\PyPoly.pyx because it changed.
  [1/4] Cythonizing Pyfhel\PyCtxt.pyx
  Traceback (most recent call last):
    File "D:\Environment\Anaconda\envs\pycrcnn\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "D:\Environment\Anaconda\envs\pycrcnn\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "D:\Environment\Anaconda\envs\pycrcnn\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\kubrick\AppData\Local\Temp\pip-build-env-u39y6m7o\overlay\Lib\site-packages\setuptools\build_meta.py", line 177, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "C:\Users\kubrick\AppData\Local\Temp\pip-build-env-u39y6m7o\overlay\Lib\site-packages\setuptools\build_meta.py", line 159, in _get_build_requires
      self.run_setup()
    File "C:\Users\kubrick\AppData\Local\Temp\pip-build-env-u39y6m7o\overlay\Lib\site-packages\setuptools\build_meta.py", line 174, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 660, in <module>
      ext_modules=cythonize(
    File "C:\Users\kubrick\AppData\Local\Temp\pip-build-env-u39y6m7o\overlay\Lib\site-packages\Cython\Build\Dependencies.py", line 1134, in cythonize
      cythonize_one(*args)
    File "C:\Users\kubrick\AppData\Local\Temp\pip-build-env-u39y6m7o\overlay\Lib\site-packages\Cython\Build\Dependencies.py", line 1301, in cythonize_one
      raise CompileError(None, pyx_file)
  Cython.Compiler.Errors.CompileError: Pyfhel\PyCtxt.pyx
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'D:\Environment\Anaconda\envs\pycrcnn\python.exe' 'D:\Environment\Anaconda\envs\pycrcnn\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' get_requires_for_build_wheel 'C:\Users\kubrick\AppData\Local\Temp\tmpkxjeouz4'
  cwd: D:\Project\Python\PyCrCNN\Pyfhel
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

the problem is similar, even happen in the same line, I am wondering if I did everything correct?

Setup:

aaron0105vincent commented 1 year ago

Hi, I encountered the same problem with the latest version but the previous version installation works without any problem (i did git checkout 02decae and pip install .)

Kuuuuubrick commented 1 year ago

Thank you so much, I'll try it out!

ibarrond commented 1 year ago

My fault! I applied a hotfix without testing. I'll be mending the mistake this WE.

On Fri, 23 Jun 2023, 16:17 Kubrick, @.***> wrote:

Thank you so much, I'll try it out!

— Reply to this email directly, view it on GitHub https://github.com/ibarrond/Pyfhel/issues/203#issuecomment-1604349222, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADF6Z5YU6WCPILPPQUCUZN3XMWQOPANCNFSM6AAAAAAZNSFKMY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Kuuuuubrick commented 1 year ago

My fault! I applied a hotfix without testing. I'll be mending the mistake this WE. On Fri, 23 Jun 2023, 16:17 Kubrick, @.> wrote: Thank you so much, I'll try it out! — Reply to this email directly, view it on GitHub <#203 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADF6Z5YU6WCPILPPQUCUZN3XMWQOPANCNFSM6AAAAAAZNSFKMY . You are receiving this because you are subscribed to this thread.Message ID: @.>

Certainly, I am eagerly looking forward to your work with anticipation.

ibarrond commented 1 year ago

And it should be correct now!

Kuuuuubrick commented 1 year ago

And it should be correct now!

Thanks! Everything goes well now!