Can't install extension version or detect ARC GPU

[BrentonPoke]

I've tried installing the extension for GPU use and the only one that's found is a 0.0.0.dev1 that doesn't provide a gpu version of the extension. When I try to run my script, it then continues looking for cuda before using the CPU. I have successfully installed the other things necessary, like the oneAPI toolkit and what should be the intel-opencl driver (on Fedora, there is no intel-opencl-icd, just intel-opencl with some clang dependencies). Tensorflow is also installed and up to date, but not able to grasp my GPU.

I did try to use the docker image, but I don't think it's being launched properly even though I used the exact command in the documentation. I'm open to using it though if it makes anything easier.

Is there something wrong with the current pypi package that pip is getting? Or is there something else I need?

[yinghu5]

Hi BrentonPoke,
thank you a lot for raising the issue. we had similar issues reported before. #28 https://github.com/intel/intel-extension-for-tensorflow/issues/28. for your reference. For a quick check your system, let go to check the GPU is recognized. could you please try the command : 1) $ source /opt/intel/oneapi/setvars.sh $sycl-ls

show the GPUs are recognized or not. [opencl:cpu:0] Intel(R) OpenCL, I... [opencl:gpu:1] Intel(R) OpenCL HD Graphics, ... [opencl:cpu:2] Intel(R) OpenCL, I... [opencl:acc:3] Intel(R) ...[ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero,

If you see the GPU are ready, then please in the same environment (terminal), continue to activate your Intel extension for Tensorflow python environment. for example, I installed it in my home folder ~/miniconda3 -> itex_112 environment. execute the below command: $source ~/miniconda3/bin/activate $conda activate itex_112

then please execute the below and show your output. 2) $ cd ~/miniconda3/envs/itex_112/lib/python3.10/site-packages/intel_extension_for_tensorflow/tools $ ./env_check.sh

[BrentonPoke]

I don't think it's here. But even so, I can't install the proper version of the extension. 0.0.0.dev1 is still the version installed and doesn't match the release version.

[yinghu5]

Hi,
From the output, the good thing is that the system can recognize your HD graphics. [0x56a0][Intel Arc A770 from https://dgpu-docs.intel.com/devices/hardware-table.html]. but with OpenCL backend, not the LeverZero backend (we usually use that).

The Arc GPU support is experimental and based on Ubuntu 22.04. For Fedora, I'm not sure if it works, anyway, let's try it out. please follow the install guide: https://dgpu-docs.intel.com/releases/stable_602_20230323.html

install those packages: especially intel-igc-opencl-devel level-zero-devel intel-gsc-devel libmetee-devel \ level-zero-devel .

Then do the GPU check again: either sycl-ls or other smoke tests: https://dgpu-docs.intel.com/driver/installation.html#to-smoke-test-the-compute-stack clinfo | head -n 5.

(if you have the ITEX installation and please run ./env_check.sh and show the result, it also tells which package is missing, then install them).

[yinghu5]

About the install guide for Intel extension for Tensorflow for Arc GPU, please refer to https://github.com/intel/intel-extension-for-tensorflow/blob/main/docs/install/experimental/install_for_arc_gpu.md.

[BrentonPoke]

I installed all the packages listed and got this as output. I did notice that after running set-vars.sh the number of platforms increased from 1 to 3, though I'm not sure what that means.

[yinghu5]

Hi, thank a lot for the test. So, it seems not doable under Fedora OS, do you have other ubuntu system?

There are two kinds of runtimes supported for Arc dGPU, see details from https://github.com/intel/compute-runtime

1. Intel(R) Level-Zero
2. Intel OpenCL Intel extension for Tensorflow is using the Intel Level-Zero runtime as the backend. That is why we have to validate if the Level-zero driver is installed correctly at first.

From your output, we see the OpenCL driver, but not Level-Zero driver.

if under unbuntu 22.04 and the Level zero drivers are installed correctly, the expected output is like

(base) a770@RPLP-A770:~$ sycl-ls [opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device 1.2 [2023.15.3.0.20_160000] [opencl:cpu:1] Intel(R) OpenCL, 12th Gen Intel(R) Core(TM) i7-12700 3.0 [2023.15.3.0.20_160000] [opencl:gpu:2] Intel(R) OpenCL HD Graphics, Intel(R) Arc(TM) A770 Graphics 3.0 [23.13.26032.30] [opencl:gpu:3] Intel(R) OpenCL HD Graphics, Intel(R) UHD Graphics 770 3.0 [23.13.26032.30] [ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) Arc(TM) A770 Graphics 1.3 [1.3.26032] [ext_oneapi_level_zero:gpu:1] Intel(R) Level-Zero, Intel(R) UHD Graphics 770 1.3 [1.3.26032]

[yinghu5]

As you mentioned, docker may be another way. Intel extension for TensorFlow provides the ready Unbuntu docker:

Pull available Intel® Data Center Max GPU Series, Intel® Data Center GPU Flex Series and Xeon CPUs pre-built docker images from DockerHub directly now.

$ docker pull intel/intel-extension-for-tensorflow:gpu $ docker run -it -p 8888:8888 --device /dev/dri -v /dev/dri/by-path:/dev/dri/by-path intel/intel-extension-for-tensorflow:gpu

To use the ubuntu docker, you may still need to validate if the level zero driver is installed or not, if not, please install the necessary runtime packages: sudo apt-get install \ intel-opencl-icd=23.05.25593.18-601~22.04 \ intel-level-zero-gpu=1.3.25593.18-601~22.04 \ level-zero=1.9.4+i589~22.04 \ intel-igc-cm=1.0.176+i600~22.04 \ libigc1=1.0.13230.8-600~22.04 \ libigdfcl1=1.0.13230.8-600~22.04 \ libigdgmm12=22.3.5-601~22.04

[BrentonPoke]

The container runs and I have all those libraries and their development packages installed, but my script continues trying to find cuda libraries and sycl-ls doesn't show my GPU. I'm also still getting this from pypi: WARNING: intel-extension-for-tensorflow 0.0.0.dev1 does not provide the extra 'gpu'

[yinghu5]

Hi BrentonPoke, The WARNING: intel-extension-for-tensorflow 0.0.0.dev1 does not provide the extra 'gpu' seems be a very earlier version. you may pull the latest one with tag $docker pull intel/intel-extension-for-tensorflow:1.2.0-gpu

Anyway, in your container, could you please show the output $python -V and are you able to run the env_check.sh in the docker? root@68fe4e339c09:/# bash /usr/local/lib/python3.10/dist-packages/intel_extension_for_tensorflow/tools/env_check.sh

[BrentonPoke]

I was able to run said commands and the container has version 3.10 for python. The availability check wasn't completed though, as I get the same output about not finding cuda that I get when I try to run my own python script.

[yinghu5]

Hi, please ignore the information like "could not find cuda drivers on your machine". They were automatically prompted if no cuda devices. From the output, E itex/core/devices/gpu/itex_gpu_runtime.cc:173] can not find any devices, it is still the roadblock.

do you have the Tensorflow version and GPU driver check pass?

If the GPU driver check passed, then the message " you have multiple libstdc++.so.6" may the next rootcause of you can't run your code.

for this problem, please check if you have some of them $ find / -name libstd.so.

root@68fe4e339c09:/# find / -name libstd.so. /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30-gdb.py

one solution for it is to preload the correct one and run the env_check again:

$export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 $/usr/local/lib/python3.10/dist-packages/intel_extension_for_tensorflow/tools/env_check.sh

[BrentonPoke]

I've been trying to find that env_check.sh script in my local system, but it doesn't seem to be present after installing the tensorflow extension. Is it supposed to be installed by the pypi package? If so, There is something wrong with how pip is interacting with pypi because I can never get the latest version of the extension. Even when using pip install --no-cache-dir --upgrade intel-extension-for-tensorflow[gpu]==1.2.0, I still get that early development build.

I did try your suggestion in the container, though; nothing changed and it gave the same output. I double checked the output and the GPU drivers are installed and the check passed. I will also note that /usr/lib/x86_64-linux-gnu/libstdc++.so.6 is a symlink to libstdc++.so.6.0.30 in the container, so there shouldn't be a conflict presented by this.

[yinghu5]

Hi BrentonPoke,

Right, the env_check.sh script was installed with PYPI package by default.

As we don't support Fedora system. What one can refer to may be the OS requirement: https://dgpu-docs.intel.com/driver/os-support.html.

So it is impossible to install the latest version to your native Fedora system. (What is your Fedora kernel version and python version under the native environment? )

and in the container, you mentioned double checked the output and the GPU drivers are installed, and the check passed, Could you please copy the screen of the output of env_check.sh about tensorflow and driver versions info?

The quick solution is to try Ubuntu 22.04 system, which is the only system supported.

Thanks

[BrentonPoke]

Check Environment for Intel(R) Extension for TensorFlow*...

======================== Check Python ========================

python3.10 is installed.

==================== Check Python Passed =====================

========================== Check OS ==========================

OS ubuntu:22.04 is Supported.

====================== Check OS Passed =======================

====================== Check Tensorflow ======================

tensorflow2.12 is installed.

================== Check Tensorflow Passed ===================

=================== Check Intel GPU Driver ===================

Intel(R) graphics runtime intel-level-zero-gpu-1.3.25593.18-601 is installed. Intel(R) graphics runtime intel-opencl-icd-23.05.25593.18-601 is installed. Intel(R) graphics runtime level-zero-1.9.4+i589 is installed. Intel(R) graphics runtime libigc1-1.0.13230.8-600 is installed. Intel(R) graphics runtime libigdfcl1-1.0.13230.8-600 is installed. Intel(R) graphics runtime libigdgmm12-22.3.5-601 is installed.

=============== Check Intel GPU Driver Finshed ================

===================== Check Intel OneApi =====================

Intel(R) OneAPI DPC++/C++ Compiler is installed. Intel(R) OneAPI Math Kernel Library is installed.

================= Check Intel OneApi Passed ==================

========================== Check Devices Availability ==========================

2023-06-02 20:32:15.996216: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used. 2023-06-02 20:32:16.033079: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used. 2023-06-02 20:32:16.033491: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-06-02 20:32:16.682953: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2023-06-02 20:32:17.629292: E itex/core/devices/gpu/itex_gpu_runtime.cc:173] Can not found any devices. To check runtime environment on your host, please run itex/tools/env_check.sh. If you need help, create an issue at https://github.com/intel/intel-extension-for-tensorflow/issues You have multiple libstdc++.so.6, make sure you are using the correct one. /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30.

Enable OCL_ICD_ENABLE_TRACE=1 OCL_ICD_DEBUG=2 to obtain detail information when using ITEX.

====================== Check Devices Availability Failed =======================

[yinghu5]

Hi BrentonPoke Thank you a lot for sharing. It is a completed reference for all developers who have installation issues on Fedora.

In the container (ubuntu system), the computing runtime (green) is ok. but maybe some questions (i.e red) in other unvalidated parts.

Anyway, the last try about the error in the container.

from the output, the error E itex/core/devices/gpu/itex_gpu_runtime.cc:173] Can not found any devices:

and warning you have multiple libstdc++.so.6" may the next rootcause of you can't run your code.

for this problem, please check if you have some of them $ find / -name libstd.so.

root@68fe4e339c09:/# find / -name libstd.so. /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30-gdb.py xxx

If you have other libstd++xxx , please rm others and run the env_check again.

export OCL_ICD_ENABLE_TRACE=1 export OCL_ICD_DEBUG=2 $/usr/local/lib/python3.10/dist-packages/intel_extension_for_tensorflow/tools/env_check.sh

Thanks Ying

[BrentonPoke]

Well, my kernel is 6.2, so if it needs to be significantly lower to run on bare metal, then I guess I'll just try to get the container working from now on.

As for the libstdc++.so in the container, I only have one with a sym-link to it. Should I remove the symlink? I don't think that will do anything, will it?

[yinghu5]

Hi BrentonPoke,

Thank a lot for letting us know the result. thus as far as I can see, the current Intel extension of tensorflow [gpu] can't work with Fedora (kernel 6.2) and even the ubuntu container on it. ( may because "--device /dev/dri -v /dev/dri/by-path:/dev/dri/by-path" pass host device info to container). If it is impossible to change the OS to Unbuntu, do you have another machine with OS windows 11 (by WSL), which is in our support metrics?

(About the libstdc++, you have only one lib in your container. the symlink is ok, please just leave it there. )

Thank

[BrentonPoke]

I don't have windows on that machine, but would uninstalling the local install of oneAPI help? I'm not sure if it was ever interfering with the container.

[yinghu5]

@BrentonPoke, hope you doing well. Sorry, we failed at last step with unknown problem in docker environment. I will close the thread as no further activity. and please feel free to update if any news.

Just FYI Intel extension for TensorFlow got new release, as well as oneAPI. For everyone who want to try, please refer to https://intel.github.io/intel-extension-for-tensorflow/latest/docs/install/experimental/install_for_arc_gpu.html

[BrentonPoke]

I think an update made the GPU visible to oneAPI at some point, and I was able to get the extension installed again. Initially, DPC++ wasn't found, but I was able to install it alone (the oneAPI build for fedora doesn't seem to include it for some reason) using the script on the website. libycl.so is still missing, so I'm not sure where to go from here. But it looks like i'm almost there. Where would I go from here?

[wangkl2]

@BrentonPoke Hi, great to hear that you can now install the latest release version of ITEX. I saw two potential version mismatches from your output. [1] ITEX v2.14.0.1 works with driver 736, the expected opencl-icd version is 23.30.26918.50-736 and the corresponding output for sycl-ls opencl version is 23.30.26918.50; while the expected level-zero-gpu version is 1.3.26918.50-736 and the corresponding output for sycl-ls level-zero 1.3.26918. Could you please upgrade your driver version to 736? Please refer to https://github.com/intel/intel-extension-for-tensorflow/blob/main/docs/install/experimental/install_for_arc_gpu.md [2] "Could not load dynamic library": libsycl.so.7 ITEX v2.14.0.1 works with oneAPI 2024.0. Please help to check with icpx -v to see the version of dpcpp. If the output version is not 2024.0.0 (2024.0.0.20231017), please help to upgrade the oneAPI version to 2024.0.

[BrentonPoke]

I tried uninstalling the previous level-zero math kernel library that were from the install scripts and reinstalled the entire oneAPI basekit from here: https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html?operatingsystem=linux&distributions=dnf This should mean everything will correspond correctly, but I must note icpx -v is still an unknown command even after using the setvars.sh script. EDIT: It works now, and i get this:

Intel(R) oneAPI DPC++/C++ Compiler 2024.0.1 (2024.0.1.20231122)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/intel/oneapi/compiler/2024.0/bin/compiler
Configuration file: /opt/intel/oneapi/compiler/2024.0/bin/compiler/../icpx.cfg
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/13
Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/13
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64

I try to run my test script again and this time libze_loader.so is missing. It seems to belong to level-zero, but intel-level-zero is already installed. EDIT: It turns out i needed both intel-level-zero and oneapi-level-zero. I get this before a SIGABRT

2023-12-19 00:31:54.869214: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-12-19 00:31:55.340570: I itex/core/wrapper/itex_cpu_wrapper.cc:70] Intel Extension for Tensorflow* AVX2 CPU backend is loaded.
2023-12-19 00:31:55.827258: I itex/core/wrapper/itex_gpu_wrapper.cc:35] Intel Extension for Tensorflow* GPU backend is loaded.
2023-12-19 00:31:55.924924: I itex/core/devices/gpu/itex_gpu_runtime.cc:129] Selected platform: Intel(R) Level-Zero
2023-12-19 00:31:55.925283: I itex/core/devices/gpu/itex_gpu_runtime.cc:154] number of sub-devices is zero, expose root device.
2023-12-19 00:31:55.925840: F itex/core/utils/op_kernel.cc:54] Check failed: false Multiple KernelCreateFunc registration
If you need help, create an issue at https://github.com/intel/intel-extension-for-tensorflow/issues

sycl-ls output is as follows:

(venv) brentonpoke@crash-override:~/Documents/nbaiot$ sycl-ls
[opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device OpenCL 1.2  [2023.16.11.0.22_160000]
[opencl:cpu:1] Intel(R) OpenCL, Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz OpenCL 3.0 (Build 0) [2023.16.11.0.22_160000]
[opencl:gpu:2] Intel(R) OpenCL Graphics, Intel(R) Arc(TM) A770 Graphics OpenCL 3.0 NEO  [23.35.27191.9]
[ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) Arc(TM) A770 Graphics 1.3 [1.3.27191]

These seem to be at least the minimum you required, so shouldn't this work?

[wangkl2]

@BrentonPoke Hi, from your output, the GPU device can be detected by ITEX now. Could you please also provide the output of the env_check.sh?

export path_to_site_packages=`python -c "import site; print(site.getsitepackages()[0])"`
bash ${path_to_site_packages}/intel_extension_for_tensorflow/tools/env_check.sh

Also, could you please share the workload you are running that throws the error? Thanks.

[BrentonPoke]

========================  Check Python  ========================

 python3.10 is installed. 

====================  Check Python Passed  =====================

==========================  Check OS  ==========================

 Unknow OS fedora. 

======================   Check OS Failed  =======================

The workload is a python script that reads in a dataset and trains a neural net

# # IoT Intrusion Detection
# 
# The N-BaIoT Dataset contains traffic data for 9 IoT devices. The data comprise of both benign traffic and of a variety of malicious attacks. Here we run three deep neural networks to identify cyberattacks on a Provision PT-737E Security Camera.

# %% [code] {"jupyter":{"outputs_hidden":false}}
import datetime, ray
import numpy as np
import pandas as pd
ray.init()
# %% [code] {"jupyter":{"outputs_hidden":false}}
benign=pd.read_csv('input/8.benign.csv')
g_c=pd.read_csv('input/8.gafgyt.combo.csv')
g_j=pd.read_csv('input/8.gafgyt.junk.csv')
g_s=pd.read_csv('input/8.gafgyt.scan.csv')
g_t=pd.read_csv('input/8.gafgyt.tcp.csv')
g_u=pd.read_csv('input/8.gafgyt.udp.csv')
ack=pd.read_csv('input/8.mirai.ack.csv')
sca=pd.read_csv('input/8.mirai.scan.csv')
syn=pd.read_csv('input/8.mirai.syn.csv')
udp=pd.read_csv('input/8.mirai.udp.csv')
pln=pd.read_csv('input/8.mirai.udpplain.csv')

benign7 = pd.read_csv('input/7.benign.csv')
g_c7=pd.read_csv('input/7.gafgyt.combo.csv')
g_j7=pd.read_csv('input/7.gafgyt.junk.csv')
g_s7=pd.read_csv('input/7.gafgyt.scan.csv')
g_t7=pd.read_csv('input/7.gafgyt.tcp.csv')
g_u7=pd.read_csv('input/7.gafgyt.udp.csv')

benign3 = pd.read_csv('input/3.benign.csv')
g_c3=pd.read_csv('input/3.gafgyt.combo.csv')
g_j3=pd.read_csv('input/3.gafgyt.junk.csv')
g_s3=pd.read_csv('input/3.gafgyt.scan.csv')
g_t3=pd.read_csv('input/3.gafgyt.tcp.csv')
g_u3=pd.read_csv('input/3.gafgyt.udp.csv')

benign1 = pd.read_csv('input/1.benign.csv')
g_c1=pd.read_csv('input/1.gafgyt.combo.csv')
g_j1=pd.read_csv('input/1.gafgyt.junk.csv')
g_s1=pd.read_csv('input/1.gafgyt.scan.csv')
g_t1=pd.read_csv('input/1.gafgyt.tcp.csv')
g_u1=pd.read_csv('input/1.gafgyt.udp.csv')
ack1=pd.read_csv('input/1.mirai.ack.csv')
sca1=pd.read_csv('input/1.mirai.scan.csv')
syn1=pd.read_csv('input/1.mirai.syn.csv')
udp1=pd.read_csv('input/1.mirai.udp.csv')
pln1=pd.read_csv('input/1.mirai.udpplain.csv')

benign4 = pd.read_csv('input/4.benign.csv')
g_c4=pd.read_csv('input/4.gafgyt.combo.csv')
g_j4=pd.read_csv('input/4.gafgyt.junk.csv')
g_s4=pd.read_csv('input/4.gafgyt.scan.csv')
g_t4=pd.read_csv('input/4.gafgyt.tcp.csv')
g_u4=pd.read_csv('input/4.gafgyt.udp.csv')
ack4=pd.read_csv('input/4.mirai.ack.csv')
sca4=pd.read_csv('input/4.mirai.scan.csv')
syn4=pd.read_csv('input/4.mirai.syn.csv')
udp4=pd.read_csv('input/4.mirai.udp.csv')
pln4=pd.read_csv('input/4.mirai.udpplain.csv')

benign2 = pd.read_csv('input/2.benign.csv')
g_c2=pd.read_csv('input/2.gafgyt.combo.csv')
g_j2=pd.read_csv('input/2.gafgyt.junk.csv')
g_s2=pd.read_csv('input/2.gafgyt.scan.csv')
g_t2=pd.read_csv('input/2.gafgyt.tcp.csv')
g_u2=pd.read_csv('input/2.gafgyt.udp.csv')
ack2=pd.read_csv('input/2.mirai.ack.csv')
sca2=pd.read_csv('input/2.mirai.scan.csv')
syn2=pd.read_csv('input/2.mirai.syn.csv')
udp2=pd.read_csv('input/2.mirai.udp.csv')
pln2=pd.read_csv('input/2.mirai.udpplain.csv')

benign6 = pd.read_csv('input/6.benign.csv')
g_c6=pd.read_csv('input/6.gafgyt.combo.csv')
g_j6=pd.read_csv('input/6.gafgyt.junk.csv')
g_s6=pd.read_csv('input/6.gafgyt.scan.csv')
g_t6=pd.read_csv('input/6.gafgyt.tcp.csv')
g_u6=pd.read_csv('input/6.gafgyt.udp.csv')
ack6=pd.read_csv('input/6.mirai.ack.csv')
sca6=pd.read_csv('input/6.mirai.scan.csv')
syn6=pd.read_csv('input/6.mirai.syn.csv')
udp6=pd.read_csv('input/6.mirai.udp.csv')
pln6=pd.read_csv('input/6.mirai.udpplain.csv')

benign9 = pd.read_csv('input/9.benign.csv')
g_c9=pd.read_csv('input/9.gafgyt.combo.csv')
g_j9=pd.read_csv('input/9.gafgyt.junk.csv')
g_s9=pd.read_csv('input/9.gafgyt.scan.csv')
g_t9=pd.read_csv('input/9.gafgyt.tcp.csv')
g_u9=pd.read_csv('input/9.gafgyt.udp.csv')
ack9=pd.read_csv('input/9.mirai.ack.csv')
sca9=pd.read_csv('input/9.mirai.scan.csv')
syn9=pd.read_csv('input/9.mirai.syn.csv')
udp9=pd.read_csv('input/9.mirai.udp.csv')
pln9=pd.read_csv('input/9.mirai.udpplain.csv')

frames  = [g_c,g_c7,g_c1,g_c2,g_c6,g_c9,g_c3,g_c4]
frames1 = [g_j,g_j7,g_j1,g_j2,g_j6,g_j9,g_j3,g_j4]
frames2 = [g_s,g_s7,g_s1,g_s2,g_s6,g_s9,g_s3,g_s4]
frames3 = [g_t,g_t7,g_t1,g_t2,g_t6,g_t9,g_t3,g_t4]
frames4 = [g_u,g_u7,g_u1,g_u2,g_u6,g_u9,g_u3,g_u4]
frames5 = [ack,ack2,ack1,ack6,ack9,ack4]
frames6 = [sca,sca2,sca1,sca6,sca9,sca4]
frames7 = [syn,syn2,syn1,syn6,syn9,syn4]
frames8 = [udp,udp2,udp1,udp6,udp9,udp4]
frames9 = [pln,pln2,pln1,pln6,pln9,pln4]
frames0 = [benign,benign7,benign1,benign2,benign6,benign9,benign3,benign4]

benign = pd.concat(frames0)
g_c = pd.concat(frames)
g_j = pd.concat(frames1)
g_s = pd.concat(frames2)
g_t = pd.concat(frames3)
g_u = pd.concat(frames4)
ack = pd.concat(frames5)
sca = pd.concat(frames6)
syn = pd.concat(frames7)
udp = pd.concat(frames8)
pln = pd.concat(frames9)
benign=benign.sample(frac=0.25,replace=False)
g_c=g_c.sample(frac=0.25,replace=False)
g_j=g_j.sample(frac=0.5,replace=False)
g_s=g_s.sample(frac=0.5,replace=False)
g_t=g_t.sample(frac=0.15,replace=False)
g_u=g_u.sample(frac=0.15,replace=False)
ack = ack.sample(frac=0.15,replace=False)
sca = sca.sample(frac=0.15,replace=False)
syn = syn.sample(frac=0.15,replace=False)
udp = udp.sample(frac=0.15,replace=False)
pln = pln.sample(frac=0.15,replace=False)

benign['type']='benign'
g_c['type']='gafgyt_combo'
g_j['type']='gafgyt_junk'
g_s['type']='gafgyt_scan'
g_t['type']='gafgyt_tcp'
g_u['type']='gafgyt_udp'
ack['type']='mirai_ack'
sca['type']='mirai_scan'
syn['type']='mirai_syn'
udp['type']='mirai_udp'
pln['type']='mirai_udpplain'

data=pd.concat([benign,g_c,g_j,g_s,g_t,g_u,ack,sca,syn,udp,pln],
               axis=0, sort=False, ignore_index=True)

# %% [code] {"jupyter":{"outputs_hidden":false}}
#how many instances of each class
data.groupby('type')['type'].count()

# %% [code] {"jupyter":{"outputs_hidden":false}}
#shuffle rows of dataframe 
sampler=np.random.permutation(len(data))
data=data.take(sampler)
data.head(20)

# %% [code] {"jupyter":{"outputs_hidden":false}}
#dummy encode labels, store separately
labels_full=pd.get_dummies(data['type'], prefix='type')
labels_full.head()

# %% [code] {"jupyter":{"outputs_hidden":false}}
#drop labels from training dataset
data=data.drop(columns='type')
data.head()

# %% [code] {"jupyter":{"outputs_hidden":false}}
#standardize numerical columns
def standardize(df,col):
    df[col]= (df[col]-df[col].mean())/df[col].std()

data_st=data.copy()
for i in (data_st.iloc[:,:-1].columns):
    standardize (data_st,i)

data_st.head()

# %% [code] {"jupyter":{"outputs_hidden":false}}
#training data for the neural net
train_data_st=data_st.values
train_data_st

# %% [code] {"jupyter":{"outputs_hidden":false}}
#labels for training
labels=labels_full.values
labels

# %% [markdown]
# ### Keras model

# %% [code] {"jupyter":{"outputs_hidden":false}}
#import libraries
import tensorflow as tf
from sklearn.model_selection import train_test_split
from sklearn import metrics
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation
from tensorflow.keras.callbacks import EarlyStopping

# test/train split  25% test
x_train_st, x_test_st, y_train_st, y_test_st = train_test_split(
    train_data_st, labels, test_size=0.25, random_state=42)

log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)

#  create and fit model
# model = Sequential()
# model.add(Dense(10, input_dim=train_data_st.shape[1], activation='relu'))
# model.add(Dense(40, input_dim=train_data_st.shape[1], activation='relu'))
# model.add(Dense(10, input_dim=train_data_st.shape[1], activation='relu'))
# model.add(Dense(5, input_dim=train_data_st.shape[1], activation='relu'))
# model.add(Dense(1, kernel_initializer='normal'))
# model.add(Dense(labels.shape[1],activation='softmax'))
# model.compile(loss='categorical_crossentropy', optimizer='adam')
# monitor = EarlyStopping(monitor='val_loss', min_delta=1e-3,
#                         patience=5, verbose=1, mode='auto')
# model.fit(x_train_st,y_train_st,validation_data=(x_test_st,y_test_st),
#           callbacks=[monitor, tensorboard_callback],verbose=2,epochs=50)
#
# # %% [code] {"jupyter":{"outputs_hidden":false}}
# # metrics
# pred_st = model.predict(x_test_st)
# pred_st = np.argmax(pred_st,axis=1)
# y_eval_st = np.argmax(y_test_st,axis=1)
# score_st = metrics.accuracy_score(y_eval_st, pred_st)
# print("accuracy: {}".format(score_st))

# %% [code] {"jupyter":{"outputs_hidden":false}}
#second model
model2 = Sequential()
model2.add(Dense(32, input_dim=train_data_st.shape[1], activation='relu'))
model2.add(Dense(72, input_dim=train_data_st.shape[1], activation='relu'))
model2.add(Dense(32, input_dim=train_data_st.shape[1], activation='relu'))
model2.add(Dense(8, input_dim=train_data_st.shape[1], activation='relu'))
model2.add(Dense(1, kernel_initializer='normal'))
model2.add(Dense(labels.shape[1],activation='softmax'))
model2.compile(loss='categorical_crossentropy', optimizer='adam')
monitor = EarlyStopping(monitor='val_loss', min_delta=1e-3,
                        patience=5, verbose=1, mode='auto')
model2.fit(x_train_st,y_train_st,validation_data=(x_test_st,y_test_st),
          callbacks=[monitor, tensorboard_callback], verbose=2, epochs=100)

# %% [code] {"jupyter":{"outputs_hidden":false}}
# metrics
pred_st1 = model2.predict(x_test_st)
pred_st1 = np.argmax(pred_st1,axis=1)
y_eval_st1 = np.argmax(y_test_st,axis=1)
score_st1 = metrics.accuracy_score(y_eval_st1, pred_st1)
print("accuracy: {}".format(score_st1))```

[guizili0]

@BrentonPoke can you share your pip list result or can you use a clean conda env? from the error log, seems you have install multiple version extesion.

2023-12-19 00:31:55.925840: F itex/core/utils/op_kernel.cc:54] Check failed: false Multiple KernelCreateFunc registration

[BrentonPoke]

Package                            Version
---------------------------------- ------------
absl-py                            1.4.0
aiohttp                            3.9.1
aiohttp-cors                       0.7.0
aiosignal                          1.3.1
astunparse                         1.6.3
async-timeout                      4.0.3
attrs                              23.1.0
blessed                            1.20.0
cachetools                         5.3.2
certifi                            2023.11.17
charset-normalizer                 3.3.2
click                              8.1.7
colorful                           0.5.5
contourpy                          1.2.0
cycler                             0.12.1
distlib                            0.3.8
docopt                             0.6.2
filelock                           3.13.1
flatbuffers                        23.5.26
fonttools                          4.46.0
frozenlist                         1.4.1
fsspec                             2023.12.2
gast                               0.5.4
google-api-core                    2.15.0
google-auth                        2.25.2
google-auth-oauthlib               1.0.0
google-pasta                       0.2.0
googleapis-common-protos           1.62.0
gpustat                            1.1.1
grpcio                             1.60.0
h5py                               3.10.0
idna                               3.6
intel-extension-for-tensorflow     2.14.0.1
intel-extension-for-tensorflow-lib 2.14.0.1.2
joblib                             1.3.2
jsonschema                         4.20.0
jsonschema-specifications          2023.11.2
keras                              2.14.0
kiwisolver                         1.4.5
libclang                           16.0.6
Markdown                           3.5.1
MarkupSafe                         2.1.3
matplotlib                         3.8.2
ml-dtypes                          0.2.0
modin                              0.26.0
msgpack                            1.0.7
multidict                          6.0.4
numpy                              1.23.5
nvidia-ml-py                       12.535.133
oauthlib                           3.2.2
opencensus                         0.11.3
opencensus-context                 0.1.3
opt-einsum                         3.3.0
packaging                          23.2
pandas                             2.1.4
Pillow                             10.1.0
pip                                23.2.1
pipreqs                            0.4.13
platformdirs                       3.11.0
prometheus-client                  0.19.0
protobuf                           4.23.4
psutil                             5.9.7
py-spy                             0.3.14
pyasn1                             0.5.1
pyasn1-modules                     0.3.0
pydantic                           1.10.13
pyparsing                          3.1.1
python-dateutil                    2.8.2
pytz                               2023.3.post1
PyYAML                             6.0.1
ray                                2.8.1
referencing                        0.32.0
requests                           2.31.0
requests-oauthlib                  1.3.1
rpds-py                            0.14.1
rsa                                4.9
scikit-learn                       1.3.2
scipy                              1.11.4
setuptools                         67.7.2
six                                1.16.0
smart-open                         6.4.0
tensorboard                        2.14.1
tensorboard-data-server            0.7.2
tensorflow                         2.14.1
tensorflow-estimator               2.14.0
tensorflow-io-gcs-filesystem       0.34.0
termcolor                          2.4.0
threadpoolctl                      3.2.0
typing_extensions                  4.9.0
tzdata                             2023.3
urllib3                            2.1.0
virtualenv                         20.21.0
wcwidth                            0.2.12
Werkzeug                           3.0.1
wheel                              0.42.0
wrapt                              1.14.1
yarg                               0.1.9
yarl                               1.9.4

[guizili0]

@BrentonPoke please try to remove intel-extension-for-tensorflow-lib and intel-extension-for-tensorflow And then install intel-extension-for-tensorflow[gpu]

[BrentonPoke]

Doesn't the extension use [xpu] now? Or is that different?

[guizili0]

No, we are using XPU, but did not test on Fedora. GPU whl is clean, just for debug.

[BrentonPoke]

Still not working, but now I get this:

2023-12-21 21:55:15.596518: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-12-21 21:55:16.834387: I itex/core/wrapper/itex_gpu_wrapper.cc:35] Intel Extension for Tensorflow* GPU backend is loaded.
2023-12-21 21:55:16.931244: I itex/core/devices/gpu/itex_gpu_runtime.cc:129] Selected platform: Intel(R) Level-Zero
2023-12-21 21:55:16.931616: I itex/core/devices/gpu/itex_gpu_runtime.cc:154] number of sub-devices is zero, expose root device.
2023-12-21 21:55:16.932177: F tensorflow/c/experimental/stream_executor/stream_executor.cc:751] Non-OK-status: stream_executor::MultiPlatformManager::RegisterPlatform( std::move(cplatform)) status: INTERNAL: platform is already registered with name: "XPU"

There is also a core dump that happens right after.

[guizili0]

@BrentonPoke I still suspect your env has more extension. Can you help to share the log with: strace -e trace=open,openat python

[BrentonPoke]

openat(AT_FDCWD, "/opt/intel/oneapi/tbb/2021.11/env/../lib/intel64/gcc4.8/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/tbb/2021.11/env/../lib/intel64/gcc4.8/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/tbb/2021.11/env/../lib/intel64/gcc4.8/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/opt/mpi/libfabric/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/opt/mpi/libfabric/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/opt/mpi/libfabric/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mkl/2024.0/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mkl/2024.0/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mkl/2024.0/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ippcp/2021.9/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ippcp/2021.9/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ippcp/2021.9/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ipp/2021.10/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ipp/2021.10/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ipp/2021.10/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dpl/2022.3/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dpl/2022.3/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dpl/2022.3/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dnnl/2024.0/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dnnl/2024.0/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dnnl/2024.0/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/debugger/2024.0/opt/debugger/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/debugger/2024.0/opt/debugger/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/debugger/2024.0/opt/debugger/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dal/2024.0/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dal/2024.0/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dal/2024.0/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/oclfpga/host/linux64/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/oclfpga/host/linux64/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/oclfpga/host/linux64/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/compiler/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/compiler/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/compiler/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ccl/2021.11/lib/glibc-hwcaps/x86-64-v3/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ccl/2021.11/lib/glibc-hwcaps/x86-64-v2/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ccl/2021.11/lib/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libpython3.10.so.1.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/intel/oneapi/tbb/2021.11/env/../lib/intel64/gcc4.8/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/opt/mpi/libfabric/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mkl/2024.0/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ippcp/2021.9/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ipp/2021.10/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dpl/2022.3/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dnnl/2024.0/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/debugger/2024.0/opt/debugger/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dal/2024.0/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/oclfpga/host/linux64/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/compiler/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ccl/2021.11/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/intel/oneapi/tbb/2021.11/env/../lib/intel64/gcc4.8/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/opt/mpi/libfabric/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mpi/2021.11/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/mkl/2024.0/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ippcp/2021.9/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ipp/2021.10/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dpl/2022.3/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dnnl/2024.0/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/debugger/2024.0/opt/debugger/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/dal/2024.0/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/oclfpga/host/linux64/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/opt/compiler/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/compiler/2024.0/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/intel/oneapi/ccl/2021.11/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/gconv/gconv-modules.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/bin/pyvenv.cfg", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/pyvenv.cfg", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/bin/pybuilddir.txt", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/localtime", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/intel/oneapi/advisor/2024.0/pythonapi", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/encodings/__pycache__/__init__.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/codecs.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/encodings", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/encodings/__pycache__/aliases.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/encodings/__pycache__/utf_8.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/io.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/abc.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/site.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/os.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/stat.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/_collections_abc.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/posixpath.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/genericpath.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/__pycache__/_sitebuiltins.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/pyvenv.cfg", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib64/python3.10/site-packages", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib64/python3.10/site-packages/distutils-precedence.pth", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib64/python3.10/lib-dynload", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib64/python3.10/site-packages", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib64/python3.10/site-packages/_distutils_hack/__pycache__/__init__.cpython-310.pyc", O_RDONLY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib64/python3.10/site-packages/modin-autoimport-pandas.pth", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib/python3.10/site-packages", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib/python3.10/site-packages/distutils-precedence.pth", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib/python3.10/site-packages/modin-autoimport-pandas.pth", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib64/python3.10/site-packages", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib64/python3.10/site-packages/distutils-precedence.pth", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib64/python3.10/site-packages/modin-autoimport-pandas.pth", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib/python3.10/site-packages", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib/python3.10/site-packages/distutils-precedence.pth", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib/python3.10/site-packages/modin-autoimport-pandas.pth", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/brentonpoke/Documents/nbaiot/venv/lib/python3.10/site-packages", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
+++ exited with 0 +++

Is there any way to unregister a platform and start over? I previously tried to build the extension from source, so i'm not sure if that's the conflict.

[guizili0]

@BrentonPoke Yes, that is my guess, so I try to use this trace log to find the duplicate itex so file, then remove the wrong one, but from your log, seems do not load itex. Do you meet issue when you get this log?