OFS / opae-sdk

Open Programmable Acceleration Engine
https://ofs.github.io
BSD 3-Clause "New" or "Revised" License
258 stars 82 forks source link

can it run on arm /aarch64/ armv8l? #2613

Closed AndreV84 closed 3 weeks ago

AndreV84 commented 2 years ago

it won't build on arm64

/opae-sdk-2.1.0-1/libraries/plugins/vfio/opae_vfio.c:664:2: error: implicit declaration of function ‘__builtin_cpu_init’; did you mean ‘__builtin_irint’? [-Werror=implicit-function-declaration]
  664 |  __builtin_cpu_init();
      |  ^~~~~~~~~~~~~~~~~~
      |  __builtin_irint
tswhison commented 2 years ago

Hello, it seems that __builtin_cpuinit() is not available on your platform. It seems we should add a preprocessor check for __arm_\, perhaps?

AndreV84 commented 2 years ago

@tswhison how can it be added?

tswhison commented 2 years ago

Could you try this branch? #2616

AndreV84 commented 2 years ago

curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null sudo apt update sudo apt install gh


`~/opae-sdk$ gh pr checkout 2616
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 8 (delta 6), reused 8 (delta 6), pack-reused 0
Unpacking objects: 100% (8/8), 845 bytes | 70.00 KiB/s, done.
From https://github.com/OPAE/opae-sdk
 * [new branch]        tswhison/preproc -> origin/tswhison/preproc
Branch 'tswhison/preproc' set up to track remote branch 'tswhison/preproc' from 'origin'.
Switched to a new branch 'tswhison/preproc'
`

`32%] Built target board_a10gx
[ 32%] Building C object libraries/plugins/xfpga/CMakeFiles/xfpga.dir/hostif.c.o
/tmp/ccA665pN.s: Assembler messages:
/tmp/ccA665pN.s:2639: Error: unknown mnemonic `vmovdqu64' -- `vmovdqu64 (x24),%zmm0'
/tmp/ccA665pN.s:2639: Error: unknown mnemonic `vmovdqu64' -- `vmovdqu64 %zmm0,(x21)'
make[2]: *** [libraries/plugins/xfpga/CMakeFiles/xfpga.dir/build.make:188: libraries/plugins/xfpga/CMakeFiles/xfpga.dir/mmio.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 32%] Building CXX object binaries/fpgadiag/CMakeFiles/opae-c++-nlb.dir/src/nlb_cache_prime.cpp.o
[ 33%] Building C object libraries/libboard/board_common/CMakeFiles/board_common.dir/__/__/__/tests/framework/mock/opae_std.c.o
[ 33%] Building CXX object binaries/fpgadiag/CMakeFiles/opae-c++-nlb.dir/src/nlb_stats.cpp.o
[ 34%] Building CXX object binaries/fpgadiag/CMakeFiles/opae-c++-nlb.dir/src/nlb0.cpp.o
[ 34%] Linking C static library ../../../lib/libboard_common.a
[ 35%] Built target board_common
[ 35%] Building CXX object binaries/fpgadiag/CMakeFiles/eth_group.dir/src/eth_group.cpp.o
make[1]: *** [CMakeFiles/Makefile2:1719: libraries/plugins/xfpga/CMakeFiles/xfpga.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 35%] Building CXX object binaries/fpgadiag/CMakeFiles/opae-c++-nlb.dir/src/nlb3.cpp.o
/tmp/ccqQ69yz.s: Assembler messages:
/tmp/ccqQ69yz.s:8616: Error: unknown mnemonic `vmovdqu64' -- `vmovdqu64 (x21),%zmm0'
/tmp/ccqQ69yz.s:8616: Error: unknown mnemonic `vmovdqu64' -- `vmovdqu64 %zmm0,(x0)'
make[2]: *** [libraries/plugins/vfio/CMakeFiles/opae-v.dir/build.make:90: libraries/plugins/vfio/CMakeFiles/opae-v.dir/opae_vfio.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1773: libraries/plugins/vfio/CMakeFiles/opae-v.dir/all] Error 2
[ 36%] Building CXX object binaries/fpgadiag/CMakeFiles/opae-c++-nlb.dir/src/nlb7.cpp.o
[ 36%] Linking C static library ../../lib/libopae-c-archive.a
[ 36%] Built target opae-c-archive
[ 36%] Building CXX object binaries/fpgadiag/CMakeFiles/opae-c++-nlb.dir/src/perf_counters.cpp.o
[ 36%] Building CXX object binaries/fpgadiag/CMakeFiles/opae-c++-nlb.dir/src/diag_utils.cpp.o
[ 37%] Linking CXX shared library ../../lib/libopae-c++-nlb.so
[ 37%] Built target opae-c++-nlb
[ 37%] Linking CXX shared module ../../lib/pyopaeuio.cpython-38-aarch64-linux-gnu.so
[ 37%] Built target pyopaeuio
[ 38%] Linking CXX shared module ../../lib/eth_group.cpython-38-aarch64-linux-gnu.so
[ 38%] Built target eth_group
make: *** [Makefile:156: all] Error 2

`
AndreV84 commented 2 years ago

@tswhison Error: unknown mnemonicvmovdqu64' -- vmovdqu64 (x21),%zmm0' seems like in https://github.com/google/or-tools/issues/820#issuecomment-414224256

tswhison commented 2 years ago

Ah yes. Thank you. I don't have an ARM test bench. We wouldn't be able to compile those instructions for non-x86 I think. I have updated the branch.

AndreV84 commented 2 years ago

maybe we can somehow patch the code to pass ther err, e.g. as in https://github.com/google/snappy/blob/main/snappy-stubs-internal.h#L70 ?

tswhison commented 2 years ago

Right now, the code will return FPGA_NOT_SUPPORTED for the 512-bit calls on arm. Is that what you mean?

AndreV84 commented 2 years ago

not exactly in my situation I was getting this error trying to build on aarch64 35%] Building CXX object binaries/fpgadiag/CMakeFiles/opae-c++-nlb.dir/src/nlb3.cpp.o /tmp/ccqQ69yz.s: Assembler messages: /tmp/ccqQ69yz.s:8616: Error: unknown mnemonicvmovdqu64' -- vmovdqu64 (x21),%zmm0' /tmp/ccqQ69yz.s:8616: Error: unknown mnemonicvmovdqu64' -- vmovdqu64 %zmm0,(x0)' make[2]: *** [libraries/plugins/vfio/CMakeFiles/opae-v.dir/build.make:90: libraries/plugins/vfio/CMakeFiles/opae-v.dir/opae_vfio.c.o] Error 1 make[1]: *** [CMakeFiles/Makefile2:1773: libraries/plugins/vfio/CMakeFiles/opae-v.dir/all] Error 2'

tswhison commented 2 years ago

I thought I had fixed that one. Could you try updating your branch for #2616?

AndreV84 commented 2 years ago

@tswhison Thank you for the update! Seems I can build it on my aarch64 device is there a simple test/ to try the board with it? https://www.intel.com/content/dam/www/central-libraries/us/en/documents/intel-pac-with-arria-10-gx-product-brief.pdf

tswhison commented 2 years ago

In order to run something, you'll need a driver so that the bit image can be configured. Please check at linux-dfl for the availability of a driver.

AndreV84 commented 2 years ago

could you extend what needs to be done after executing the step below, please? git clone https://github.com/OPAE/linux-dfl

tswhison commented 2 years ago

You'll need to check with the folks over at linux-dfl. I'm not sure whether an ARM driver exists, but they would know for certain.

AndreV84 commented 2 years ago

@tswhison could you advise also how to build these on x86_64 that got installed intel dev stack environment but doesn't seem to have built sample binaries for opae-sdk

to build these guys?

usr/src/opae/samples$ ls
hello_events.c hello_fpga.c
tswhison commented 2 years ago

Please find build instructions for hello_fpga here: https://opae.github.io/0.13.0/docs/fpga_api/quick_start/readme.html#building-a-sample-application

AndreV84 commented 2 years ago

found https://www.intel.com/content/www/us/en/docs/programmable/683633/1-2-1/running-the-hello-fpga-example.html thank you very much

AndreV84 commented 2 years ago

@tswhison do IO have to install linux-dfl if I used to install the dev stack

after installing the dev stack

after sourcing

. /home/$USER/inteldevstack/init_env.sh 
 sudo fpgasupdate $OPAE_PLATFORM_ROOT/hw/samples/nlb_mode_0/bin/\
 nlb_mode_0_unsigned.gbs
[2022-06-09 21:38:42,635] [ERROR   ] No suitable PAC found.
sudo ./hello_fpga 
Using OPAE C library version '1.1.2' build 'fa90f2cd'
Error enumerating accelerators: no driver available
Error no matching accelerator: not found
AndreV84 commented 2 years ago

[ 31.135966] dfl-fme-bridge dfl-fme-bridge.1.auto: fpga bridge [DFL FPGA FME Bridge] registered

AndreV84 commented 2 years ago
07:00.0 Processing accelerators: Intel Corporation Device 09c4 (rev ff)
AndreV84 commented 2 years ago

on 20.04 ubuntu the intel dev stack script won't run will opae-sdk/ driver build from source?

AndreV84 commented 2 years ago

@tswhison trying to build linux-dfl according to opae-sdk tutorial on x86_64 20.04 ubuntu ] fs/jffs2/scan.o CC [M] net/mac80211/vht.o CC [M] drivers/leds/leds-sgm3140.o CC drivers/acpi/acpica/exfield.o gcc: fatal error: Killed signal terminated program cc1 compilation terminated. gcc: fatal error: Killed signal terminated program cc1 compilation terminated. gcc: fatal error: Killed signal terminated program cc1 compilation terminated. gcc: fatal error: Killed signal terminated program cc1 compilation terminated. CC [M] drivers/video/backlight/88pm860x_bl.o

tswhison commented 2 years ago

Could you please raise this at linux-dfl? They should be able to help with driver issues.

nathanborak commented 3 weeks ago

Closing due to inactivity/age, and because the original question was answered. If you still require assistance, please re-open the issue.