search

intel / linux-sgx

Intel SGX for Linux*
https://www.intel.com/content/www/us/en/developer/tools/software-guard-extensions/linux-overview.html
Other
1.33k stars 543 forks source link

Building for an unsupported OS #184

Closed kaleb-himes closed 6 years ago

kaleb-himes commented 6 years ago

Hi,

I realize Debian 8.7.0 "Jessie" is NOT a supported OS for Intel SGX.

Due to various NDA agreements I can not disclose much about the project on this public forum however after speaking with our contact at Intel I was told the dev team preferred a case be opened here.

For various reasons we would like to know if it is possible to get an Enclave running on the following OE without making any changes to the kernel:

CHIPSET: Intel(R) Xeon(R) CPU E3-1270 v6 OS: Debian 8.7.0

I have been following the guide here: https://download.01.org/intel-sgx/linux-2.0/docs/Intel_SGX_Installation_Guide_Linux_2.0_Open_Source.pdf

I am working with these versions of the driver, psw and sdk as we have already worked with these in Ubuntu successfully: sgx_linux_ubuntu16.04.1_x64_psw_1.8.100.37689.bin sgx_linux_ubuntu16.04.1_x64_sdk_1.8.100.37689.bin sgx_linux_x64_driver_4b57088.bin

I have the driver installed and working, I have the option to enable SGX in BIOS. However when installing the psw I get a missing driver error in /opt/Intel/iclsClient/log/iclsClient.log for MEI and HECI as seen below:

2017-11-28 14:19:13:028 00007f1acc992700 iclsClient:CreateMeiAccess: [INF] [N/A] DETECTED_PLATFORM: 0x0c [SKL]
2017-11-28 14:19:13:028 00007f1acc992700 iclsClient:CreateHeciCaller: [DBG] CreatingHeciCaller. Config addr: 24417280
2017-11-28 14:19:13:028 00007f1acc992700 iclsClient:CreateHeciCaller: [INF] [N/A] Using MalCommandLocal
2017-11-28 14:19:13:030 00007f1acc992700 iclsClient:initializeMeiAccessSingleton: [ERR] [N/A] Exception occurred when initializing MeiAccess singleton=Both MEI and HECI device driver's are missing.:
2017-11-28 14:19:13:030 00007f1acc992700 iclsClient:iclsInit: [ERR] iclsInit: initializeMeiAccessSingleton returned error. Status: STATUS_ERROR.

In searching for a solution to installing the missing AMT drivers I came across a couple of sources that indicated I may need to recompile the kernel to achieve this. Part of the project requirements specify that the kernel must be 8.7.0 so doing this would conflict with that requirement in effect updating to Debian 8.7.0-recompiled-with-AMT. This is not allowed.

Will it be possible for me to get Intel SGX running on Debian 8.7.0?

Warmest Regards,

Kaleb

bodzhang commented 6 years ago

MEI/HECI and iclsClient are only needed to support the SGX Platform Service (monotonic counter and time - https://software.intel.com/sites/default/files/managed/1b/a2/Intel-SGX-Platform-Services.pdf). Does your usage need those services?

kaleb-himes commented 6 years ago

Hi @bodzhang,

No we do not have a need to enforce expiration dates or playback count limits on any license policies for this project. Can I safely proceed with installation of the sdk after the psw install has reported:

SGX/1.8# ./sgx_linux_ubuntu16.04.1_x64_psw_1.8.100.37689.bin 
Unpacking Intel SGX PSW Package ... done.
Verifying the integrity of the install package ... done.
Installing Intel SGX PSW Package ... done.
/tmp/sgx-psw-lKxbN7 /home/cissfips/SGX/1.8
install -d /opt/intel/sgxpsw
install -d /opt/intel/sgxpsw/scripts
install -d /usr/lib
cp -r package/* /opt/intel/sgxpsw
install scripts/* /opt/intel/sgxpsw/scripts
mv /opt/intel/sgxpsw/lib64/libsgx_uae_service.so /usr/lib
mv /opt/intel/sgxpsw/lib64/libsgx_urts.so /usr/lib
rmdir /opt/intel/sgxpsw/lib64
/home/cissfips/SGX/1.8
Installing aesmd.service service ...Created symlink /etc/systemd/system/multi-user.target.wants/aesmd.service → /lib/systemd/system/aesmd.service.
 done.
iclsInit() returned error.
Trusted platform service is unavailable. Check log in /opt/Intel/iclsClient/log/iclsClient.log.
Job for aesmd.service failed because the control process exited with error code.
See "systemctl status aesmd.service" and "journalctl -xe" for details.

SGX/1.8# tail -f /opt/Intel/iclsClient/log/iclsClient.log
... Errors included in last ...
bodzhang commented 6 years ago

Probably. If the iclsclient failure indeed affects other SGX operation, uninstalling the iclsclient might be a good workaround.

kaleb-himes commented 6 years ago

I'll try that. Thank you @bodzhang

kaleb-himes commented 6 years ago

Hi @bodzhang,

That solution does not work. It appears there is a dependancy on *MEI driver regardless if I need those features or not:

-- Unit aesmd.service has finished shutting down.
Nov 28 15:25:29 ciss systemd[1]: Couldn't stat device /dev/mei0
Nov 28 15:25:29 ciss systemd[1]: Starting Intel(R) Architectural Enclave Service Manager...
-- Subject: Unit aesmd.service has begun start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- Unit aesmd.service has begun starting up.
Nov 28 15:25:29 ciss aesm_service[2909]: /opt/intel/sgxpsw/aesm/aesm_service: symbol lookup error: /opt/intel/sgxpsw/aesm/aesm_service: undefined
Nov 28 15:25:29 ciss systemd[1]: aesmd.service: Control process exited, code=exited status=127
Nov 28 15:25:29 ciss systemd[1]: Failed to start Intel(R) Architectural Enclave Service Manager.

With iclsClient uninstalled I see the above and get this error when calling aesm_service directly:

/opt/intel/sgxpsw/aesm/aesm_service 
/opt/intel/sgxpsw/aesm/aesm_service: symbol lookup error: /opt/intel/sgxpsw/aesm/aesm_service: undefined symbol: _ZNK6google8protobuf7Message11GetTypeNameB5cxx11Ev
bodzhang commented 6 years ago

https://download.01.org/intel-sgx/linux-2.0/docs/Intel_SGX_Installation_Guide_Linux_2.0_Open_Source.pdf instruction on installation steps for the optional SGX Platform Service mentions the JHI service. Can you try uninstalling the JHI service too?

kaleb-himes commented 6 years ago

Sure thing,

I had to stop work on it for the day. Will resume first thing in the morning and let you know my results. Thanks for your help so far.

haitaohuang commented 6 years ago

Error message indicates you are missing some protobuf symbols. You may have a wrong version of protobuf installed? Ubuntu 16.04 uses version 2.6.1 libprotobuf. If your env has different version, you may need replace with that version or build PSW from source.

llly commented 6 years ago

Did you build the sgx_psw installer or use the pre-built one used in Ubuntu? The protobuf version may be different between Ubuntu and Debian. You can try to build the sgx_psw installer on Debian following the instructions in README.

kaleb-himes commented 6 years ago

This item is now resolved. Apologies for the delay in closing it out.

-K

kaleb-himes commented 6 years ago

Instructions for future users looking to resolve a similar issue:

These steps have ONLY been tested on a fresh installation of Debian 8.7.0. The image used was retrieved from:

URL:

https://cdimage.debian.org/cdimage/archive/8.7.0/amd64/iso-cd/debian-8.7.0-amd64-CD-1.iso

Again, Please NOTE: this was only done with a fresh install. This solution did NOT work on a previously used system that had an updated version of GCC. The system had to be wiped clean and freshly install from a live-boot USB to get this solution to work.

The SGX SDK and Driver version used was v1.8

After performing the fresh install create a new directory in /home/ called "SGX"

cd ~/
mkdir SGX
mkdir v1.8

cd v1.8

# fetch tools necessary to build SGX from intel github repo
su root
# enter sudo / root password
apt-get update
apt-get install git
apt-get install linux-headers-$(uname -r)
apt-get install build-essential ocaml automake autoconf libtool wget python
apt-get install libssl-dev libcurl4-openssl-dev protobuf-compiler libprotobuf-dev
# return to non-root user mode
exit

# fetch sources from intel github repo
git clone https://github.com/01org/linux-sgx-driver.git
git clone https://github.com/01org/linux-sgx.git

# build/install driver
cd linux-sgx-driver
git checkout sgx_driver_1.8
make
# become root user again.
su root
mkdir -p "/lib/modules/"`uname -r`"/kernel/drivers/intel/sgx"
cp isgx.ko "/lib/modules/"`uname -r`"/kernel/drivers/intel/sgx"
cp isgx.ko "/lib/modules/"`uname -r`"/kernel/drivers/intel/sgx"
/sbin/depmod
/sbin/modprobe isgx
ls –la /dev/isgx
# return to non-root user mode
exit

# Build SDK/PSW
cd ../linux-sgx
git checkout sgx_1.8
make
make psw_install_pkg
make sdk_install_pkg
cd linux/installer/bin
# become root user again
su root
./sgx_linux_x64_psw_1.8.100.37689.bin
./sgx_linux_x64_sdk_1.8.100.37689.bin   # select no then install to ‘/opt/intel’
cp -r /opt/intel/sgxsdk/SampleCode /home/{user}/SGX/v1.8
# change ownership to non-root user
chown –R {user}:{user} /home/{user}/SGX/v1.8/SampleCode
# return to non-root user mode
exit
(OPTIONAL)
# build / run Sample Enclave

$ cd ~/SGX/v1.8/SampleCode/SampleEnclave
$ make
$ ./app

Keep in mind that if you observe a DAL error in the AESM status it is a red herring. There is no Intel ME and therefore no Platform Services on Xeon class hardware.

Some sanity checks to make sure everything is working:

git describe --tags
sgx_1.8
cd ~/SGX/v1.8/linux-sgx-driver/
git describe --tags
sgx_driver_1.8
ls -la /dev/isgx
# should see this:
# 0 crw-rw-rw- 1 root root 10, 0 Dec  5 09:26 /dev/isgx
service aesmd status
# should see this:
# aesmd.service - Intel(R) Architectural Enclave Service Manager
#   Loaded: loaded (/lib/systemd/system/aesmd.service; enabled)
#   Active: active (running) since Tue 2017-12-05 09:26:57 PST; 9min ago
#  Process: 28015 ExecStart=/opt/intel/sgxpsw/aesm/aesm_service (code=exited,
# status=0/SUCCESS)
# Main PID: 28016 (aesm_service)
#   CGroup: /system.slice/aesmd.service
#           └─28016 /opt/intel/sgxpsw/aesm/aesm_service
# 
# Dec 05 09:26:57 dtzimme-debian870 systemd[1]: Started Intel(R) Architectural Enclave Service
# Manager.
# Dec 05 09:26:57 dtzimme-debian870 aesm_service[28016]: The server sock is 0x26d5930
# Dec 05 09:26:57 dtzimme-debian870 aesm_service[28016]: [ADMIN]Platform Services initializing
# Dec 05 09:26:57 dtzimme-debian870 aesm_service[28016]: [ADMIN]Platform Services
# initialization failed due to DAL error

cd ../SampleCode/SampleEnclave/
./app
# should see this:
# Checksum(0x0x7ffc213db750, 100) = 0xfffd4143
# Info: executing thread synchronization, please wait...
# Info: SampleEnclave successfully returned.
# Enter a character before exit ...
dorflex150mg commented 5 years ago

Building on Docker Container

Hello,

I'm attempting to build for a docker container running Ubuntu 16.04. I intend on using both enclaves and Plaform Services functionality. When building in the host, I'm able to use the Platform Services. However, in the container I will always get

aesm_service[10]: [ADMIN]White List update requested aesm_service[10]: The server sock is 0x556fbfec8960 jhi[10]: JHI init at server side failed jhi[10]: JHI init failed. Status: 769

jhi[10]: JHI init at server side failed jhi[10]: JHI init failed. Status: 769

jhi[10]: JHI init at server side failed jhi[10]: JHI init failed. Status: 769

aesm_service[10]: [ADMIN]Platform Services initializing aesm_service[10]: [ADMIN]Platform Services initialization failed due to DAL error aesm_service[10]: [ADMIN]White list update request successful for Version: 46

In my icls client log I will get:

2019-01-14 16:55:58:014 00007f8ed2f43700 iclsClient:initializeMeiAccessSingleton: [ERR] [N/A] Exception occurred when initializing MeiAccess singleton=Both MEI and HECI device driver's are missing.: 2019-01-14 16:55:58:014 00007f8ed2f43700 iclsClient:iclsInit: [ERR] iclsInit: initializeMeiAccessSingleton returned error. Status: STATUS_ERROR.

both isgx and mei0 are available in /dev

I read in Intel documentation that the MEI and HECI device drivers were available in linux kernel. Am I missing some kind of linking? I'm running the container with: docker run --device /dev/isgx --device /dev/mei0 -it --privileged exp

Kind regards,

Gabriel

bodzhang commented 5 years ago

SGX Platform Service functionality also has dependency on iCLS client and DAL/JHI SW components. See https://download.01.org/intel-sgx/linux-2.4/docs/Intel_SGX_Installation_Guide_Linux_2.4_Open_Source.pdf. You container environment probably failed to access the DAL/JHI layer and the iCLS client.

andyzyb commented 5 years ago

Did you install JHI software in your container?

dorflex150mg commented 5 years ago

SGX Platform Service functionality also has dependency on iCLS client and DAL/JHI SW components. See https://download.01.org/intel-sgx/linux-2.4/docs/Intel_SGX_Installation_Guide_Linux_2.4_Open_Source.pdf. You container environment probably failed to access the DAL/JHI layer and the iCLS client.

Did you install JHI software in your container?

Yes. I have JHI. In fact, I run it (but not as a systemctl service) like so: jhid 2>&1 & It seems to work fine: jhi[156]: --> jhi start jhi[156]: <-- jhi start

When jhi is already running I get:

[ADMIN]White List update requested
aesm_service[159]: The server sock is 0x558e0833b960
jhi[156]: JHI service release prints are enabled

jhi[156]: AppletsManager::discoverVmType(), Couldn't connect to either BHv1 or BHv2.
jhi[156]: Error: discoverVmType() failed
jhi[156]: JHI init failed
jhi[159]: JHI init at server side failed
jhi[159]: JHI init failed. Status: 768

jhi[156]: JHI service release prints are enabled

jhi[156]: AppletsManager::discoverVmType(), Couldn't connect to either BHv1 or BHv2.
jhi[156]: Error: discoverVmType() failed
jhi[156]: JHI init failed
jhi[159]: JHI init at server side failed
jhi[159]: JHI init failed. Status: 768

jhi[156]: JHI service release prints are enabled

jhi[156]: AppletsManager::discoverVmType(), Couldn't connect to either BHv1 or BHv2.
jhi[156]: Error: discoverVmType() failed
jhi[156]: JHI init failed
jhi[159]: JHI init at server side failed
jhi[159]: JHI init failed. Status: 768

aesm_service[159]: [ADMIN]Platform Services initializing
aesm_service[159]: [ADMIN]Platform Services initialization failed due to DAL error
aesm_service[159]: [ADMIN]White list update request successful for Version: 48`
taoyouxian commented 4 years ago

Hi I meet the following problem when I install PSW in docker based on CentOS Linux release 7.6.1810, I wonder whether anyone meets this.

/opt/intel/sgxpsw/aesm/aesm_service: symbol lookup error: /opt/intel/sgxpsw/aesm/libipc.so: undefined symbol: _ZNK6google8protobuf11MessageLite25InitializationErrorStringB5cxx11Ev

andyzyb commented 4 years ago

@taoyouxian are you using a prebuilt installer for CentOS 7.5? CentOS 7.6 may come with a different protobuf version, so you'd better to build PSW in CentOS 7.6.

taoyouxian commented 4 years ago

@andyzyb thanks for your answer, I build PSW2.7 in CentOS Linux release 7.6.1810.

However, I modify PSW2.7 to static link protobuf library, changing file psw/uae_service/linux/Makefile, use EXTERNAL_LIB += -l:libprotobuf.a instand of EXTERNAL_LIB += -lprotobuf.

andyzyb commented 4 years ago

Looks the error came from libipc.so which is used by AESM, not uae_service.so which is used by application