occlum / occlum

Occlum is a memory-safe, multi-process library OS for Intel SGX
https://occlum.io/
Other
1.35k stars 232 forks source link

[Question] Build catboost python lib within occlum toolchain #1530

Open egorvasilevich opened 2 months ago

egorvasilevich commented 2 months ago

Summary

Hello there! I want to use catboost python lib in occlum, but it needs to be rebuilded with occlum toolchain (in case - glibc) and i have some mess with it. I tried to easyle build catboost 1.2.4 by instruction in occlum development container and it builds great! For that i additionally install clang14 and update cmake up to 3.29.2 version. Thats awesome, but i can't understand how to use glibc (based in dir /opt/occlum/glibc/lib/) in catboost build process. May be you have experience how to build projects using clang and custom glibc? May be i'm missing some clang flags in building process Much thanks for replies!

Additional packages in occlum development container, which i installed to build catboost python whl: setuptools: 69.2.0 jupyterlab: 3.6.7 conan: 1.64.0 wheel: 0.34.2-1 cmake: 3.29.2 ninja: 1.10.0 python3-dev: 3.8.2-0 lsb-release: 11.1.0 wget software-properties-common: 0.99.9.12 gnupg: 2.2.19-3 clang14

Motivation

Running python code with catboost library usage in occlum

qzheng527 commented 2 months ago

Thats awesome, but i can't understand how to use glibc (based in dir /opt/occlum/glibc/lib/) in catboost build process. @egorvasilevich You don't need use Occlum glibc to build. You just need replace the general glib libs with Occlum provided glibc libs (/opt/occlum/glibc/lib) before Occlum build. So generally the apps running in Occlum could follow below steps:

  1. Do it in Occlum development container unless you are very familiar with Occlum and SGX.
  2. Build your app in normal way or just install it ( by apt for example). BTW, golang based app should use occlum golang toolchain to build.
  3. Do occlum build, the key is to fill in required files in yaml file to enjoy the copy_bom tool for dependency auto finding and glibc replacement.
  4. Occlum run, if failed. Add OCCLUM_LOG_LEVEL=debug occlum run xxx for more information.