google-coral / pycoral

Python API for ML inferencing and transfer-learning on Coral devices
https://coral.ai
Apache License 2.0
347 stars 144 forks source link

Compilation fails on 5.19.8-arch1-1 x86/64 #90

Closed rubenkr closed 1 year ago

rubenkr commented 1 year ago

Description

Trying to install Coral on Archlinux installation. There is not working AUR package and trying to compile fails as of follows. Any help is very much appreciated !

➜ pycoral git:(master) scripts/build.sh +++ dirname scripts/build.sh ++ cd scripts ++ pwd

Click to expand! ### Issue Type Build/Install ### Operating System Linux ### Coral Device USB Accelerator ### Other Devices _No response_ ### Programming Language Python 3.5 ### Relevant Log Output ```shell ➜ pycoral git:(master) uname -a Linux T450 5.19.8-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 08 Sep 2022 18:07:42 +0000 x86_64 GNU/Linux ➜ pycoral git:(master) lspci -vvv 00:00.0 Host bridge: Intel Corporation Broadwell-U Host Bridge -OPI (rev 09) Subsystem: Lenovo Device 5034 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: bdw_uncore 00:02.0 VGA compatible controller: Intel Corporation HD Graphics 5500 (rev 09) (prog-if 00 [VGA controller]) Subsystem: Lenovo Device 5036 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: i915 Kernel modules: i915 00:03.0 Audio device: Intel Corporation Broadwell-U Audio Controller (rev 09) Subsystem: Lenovo Device 5034 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: snd_hda_intel Kernel modules: snd_hda_intel 00:14.0 USB controller: Intel Corporation Wildcat Point-LP USB xHCI Controller (rev 03) (prog-if 30 [XHCI]) Subsystem: Lenovo Device 5034 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- Kernel driver in use: xhci_hcd Kernel modules: xhci_pci 00:16.0 Communication controller: Intel Corporation Wildcat Point-LP MEI Controller #1 (rev 03) Subsystem: Lenovo Device 5034 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: mei_me Kernel modules: mei_me 00:19.0 Ethernet controller: Intel Corporation Ethernet Connection (3) I218-LM (rev 03) Subsystem: Lenovo Device 2226 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: e1000e Kernel modules: e1000e 00:1b.0 Audio device: Intel Corporation Wildcat Point-LP High Definition Audio Controller (rev 03) Subsystem: Lenovo Device 5036 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: snd_hda_intel Kernel modules: snd_hda_intel 00:1c.0 PCI bridge: Intel Corporation Wildcat Point-LP PCI Express Root Port #6 (rev e3) (prog-if 00 [Normal decode]) Subsystem: Lenovo Device 5034 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport 00:1c.1 PCI bridge: Intel Corporation Wildcat Point-LP PCI Express Root Port #3 (rev e3) (prog-if 00 [Normal decode]) Subsystem: Lenovo Device 5034 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport 00:1d.0 USB controller: Intel Corporation Wildcat Point-LP USB EHCI Controller (rev 03) (prog-if 20 [EHCI]) Subsystem: Lenovo Device 5034 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- Kernel driver in use: ehci-pci 00:1f.0 ISA bridge: Intel Corporation Wildcat Point-LP LPC Controller (rev 03) Subsystem: Lenovo Device 5034 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- SERR- Kernel driver in use: lpc_ich Kernel modules: lpc_ich 00:1f.2 SATA controller: Intel Corporation Wildcat Point-LP SATA Controller [AHCI Mode] (rev 03) (prog-if 01 [AHCI 1.0]) Subsystem: Lenovo Device 5034 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- Kernel driver in use: ahci 00:1f.3 SMBus: Intel Corporation Wildcat Point-LP SMBus Controller (rev 03) Subsystem: Lenovo Device 5034 Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- TAbort- SERR- Kernel driver in use: intel_pch_thermal Kernel modules: intel_pch_thermal 02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5227 PCI Express Card Reader (rev 01) Subsystem: Lenovo Device 5034 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: rtsx_pci Kernel modules: rtsx_pci 03:00.0 Network controller: Intel Corporation Wireless 7265 (rev 61) Subsystem: Intel Corporation Wireless-N 7265 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: iwlwifi Kernel modules: iwlwifi ➜ pycoral git:(master) lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 39 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: GenuineIntel Model name: Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz CPU family: 6 Model: 61 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 4 CPU(s) scaling MHz: 84% CPU max MHz: 2900.0000 CPU min MHz: 500.0000 BogoMIPS: 4591.43 ➜ pycoral git:(master) python --version Python 3.10.7 ```
hjonnala commented 1 year ago

Please try the steps below and try the compilation. https://github.com/google-coral/pycoral#compilation

cd pycoral

git submodule init && git submodule update
rubenkr commented 1 year ago

Thank you. I tried this with following results:


➜  pycoral git:(master) ✗ git submodule init && git submodule update
Cloning into '/home/plex/pycoral/test_data'...
Submodule path 'test_data': checked out '104342d2d3480b3e66203073dac24f4e2dbb4c41'

➜  pycoral git:(master) scripts/build.sh
+++ dirname scripts/build.sh
++ cd scripts
++ pwd
+ readonly SCRIPT_DIR=/home/plex/pycoral/scripts
+ SCRIPT_DIR=/home/plex/pycoral/scripts
+ readonly MAKEFILE=/home/plex/pycoral/scripts/../Makefile
+ MAKEFILE=/home/plex/pycoral/scripts/../Makefile
+ readonly 'DOCKER_CPUS=k8 aarch64 armv7a'
+ DOCKER_CPUS='k8 aarch64 armv7a'
+ PYTHON_VERSIONS='36 37 38 39'
+ [[ 0 -gt 0 ]]
+ for python_version in ${PYTHON_VERSIONS}
++ docker_image 36
++ case $1 in
++ echo ubuntu:18.04
+ make 'DOCKER_CPUS=k8 aarch64 armv7a' DOCKER_IMAGE=ubuntu:18.04 'DOCKER_TARGETS=pybind tflite wheel tflite-wheel' -f /home/plex/pycoral/scripts/../Makefile docker-build
/home/plex/pycoral/scripts/../Makefile:240: /home/plex/pycoral/libcoral/docker/docker.mk: No such file or directory
make: *** No rule to make target '/home/plex/pycoral/libcoral/docker/docker.mk'.  Stop.
hjonnala commented 1 year ago

please delete the pycoral folder or try the below steps in a new folder:


glaptop7:~$ git clone https://github.com/google-coral/pycoral.git
Cloning into 'pycoral'...
remote: Enumerating objects: 315, done.
remote: Total 315 (delta 0), reused 0 (delta 0), pack-reused 315
Receiving objects: 100% (315/315), 6.32 MiB | 8.47 MiB/s, done.
Resolving deltas: 100% (97/97), done.

glaptop7:~$ cd pycoral

glaptop7:~/pycoral$ git submodule init && git submodule update
Submodule 'libcoral' (https://github.com/google-coral/libcoral) registered for path 'libcoral'
Submodule 'libedgetpu' (https://github.com/google-coral/libedgetpu) registered for path 'libedgetpu'
Submodule 'test_data' (https://github.com/google-coral/test_data) registered for path 'test_data'
Cloning into '/home/hemanthreddyj/pycoral/libcoral'...
Cloning into '/home/hemanthreddyj/pycoral/libedgetpu'...
Cloning into '/home/hemanthreddyj/pycoral/test_data'...
Submodule path 'libcoral': checked out '6589d0bb49c7fdbc4194ce178d06f8cdc7b5df60'
Submodule path 'libedgetpu': checked out 'ddfa7bde33c23afd8c2892182faa3e5b4e6ad94e'
Submodule path 'test_data': checked out '104342d2d3480b3e66203073dac24f4e2dbb4c41'

glaptop7:~/pycoral$ scripts/build.sh
+++ dirname scripts/build.sh
++ cd scripts
++ pwd
+ readonly SCRIPT_DIR=/home/hemanth/pycoral/scripts
+ SCRIPT_DIR=/home/hemanth/pycoral/scripts
+ readonly MAKEFILE=/home/hemanth/pycoral/scripts/../Makefile
+ MAKEFILE=/home/hemanth/pycoral/scripts/../Makefile
+ readonly 'DOCKER_CPUS=k8 aarch64 armv7a'
+ DOCKER_CPUS='k8 aarch64 armv7a'
+ PYTHON_VERSIONS='36 37 38 39'
+ [[ 0 -gt 0 ]]
+ for python_version in ${PYTHON_VERSIONS}
++ docker_image 36
++ case $1 in
++ echo ubuntu:18.04
+ make 'DOCKER_CPUS=k8 aarch64 armv7a' DOCKER_IMAGE=ubuntu:18.04 'DOCKER_TARGETS=pybind tflite wheel tflite-wheel' -f /home/hemanthreddyj/pycoral/scripts/../Makefile docker-build
docker build  -t "coral-edgetpu-ubuntu-18.04" \
    --build-arg IMAGE=ubuntu:18.04 /home/hemanth/pycoral/libcoral/docker
Sending build context to Docker daemon  11.78kB
Step 1/14 : ARG IMAGE
Step 2/14 : FROM ${IMAGE}
rubenkr commented 1 year ago

Wow ! Thanks that looks promising. I only get now an SSL connection error. Any ideas if this is temporary? ---> 59339a8f0bbf Step 14/14 : RUN wget -O /bazel https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh && bash /bazel && rm -f /bazel ---> Running in 3eed4d740954 --2022-09-15 20:21:29-- https://github.com/bazelbuild/bazel/releases/download/4.0.0/bazel-4.0.0-installer-linux-x86_64.sh Resolving github.com (github.com)... 140.82.121.3 Connecting to github.com (github.com)|140.82.121.3|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/20773773/62d9ea00-5bc7-11eb-8a3b-314d5fb68ae7?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220915%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220915T202130Z&X-Amz-Expires=300&X-Amz-Signature=7ea680e8d47fbe9aac9e7bc3d4c60ffaefbdbd10664c40cc0e78165c9abfadfb&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=20773773&response-content-disposition=attachment%3B%20filename%3Dbazel-4.0.0-installer-linux-x86_64.sh&response-content-type=application%2Foctet-stream [following] --2022-09-15 20:21:30-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/20773773/62d9ea00-5bc7-11eb-8a3b-314d5fb68ae7?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220915%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220915T202130Z&X-Amz-Expires=300&X-Amz-Signature=7ea680e8d47fbe9aac9e7bc3d4c60ffaefbdbd10664c40cc0e78165c9abfadfb&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=20773773&response-content-disposition=attachment%3B%20filename%3Dbazel-4.0.0-installer-linux-x86_64.sh&response-content-type=application%2Foctet-stream Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.108.133, ... Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.

Unable to establish SSL connection. The command '/bin/sh -c wget -O /bazel https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh && bash /bazel && rm -f /bazel' returned a non-zero code: 4 make: *** [/home/plex/pycoral/libcoral/docker/docker.mk:31: docker-image] Error 4

hjonnala commented 1 year ago

Unable to establish SSL connection.

are you trying to build on a corporate machine or personal laptop? https://github.com/Azure/AppConfiguration/issues/244#issuecomment-696922615

rubenkr commented 1 year ago

its my personal laptop which I am using as my main server for home-assistant, frigate, etc.

Found the issue. I had wireguard running with custom DNS server. Now it's running through.

rubenkr commented 1 year ago

Unfortunately a build error has happened in the end:

Use --sandbox_debug to see verbose messages from the sandbox In file included from external/ruy/ruy/pack_arm.cc:16: external/ruy/ruy/pack_arm.h:492:9: warning: multi-line comment [-Wcomment] 492 | #endif // (RUY_PLATFORM_NEON_64 || RUY_PLATFORM_NEON_32) && \ | ^ external/ruy/ruy/pack_arm.cc: In function 'void ruy::Pack8bitColMajorForNeon4Cols(const ruy::PackParams8bit&)': external/ruy/ruy/pack_arm.cc:469:72: error: 'asm' operand has impossible constraints 469 | "q4", "q5", "q6", "q7", "q8", "q9", "q10", "q11", "q12", "q13"); | ^ Target //src:_pywrap_coral failed to build INFO: Elapsed time: 475.595s, Critical Path: 70.39s INFO: 579 processes: 173 internal, 406 processwrapper-sandbox. FAILED: Build did NOT complete successfully make: *** [Makefile:152: pybind] Error 1 make: Leaving directory '/workspace' make: *** [/home/plex/pycoral/libcoral/docker/docker.mk:40: docker-build] Error 1

hjonnala commented 1 year ago

Can you please share for which python version are you trying to build the pycoral?

rubenkr commented 1 year ago

Sure. I tried it again this night. And I think it is related to this comment here: https://aur.archlinux.org/packages/python-pycoral#comment-854595

➜ ~ python --version

Python 3.10.7

rubenkr commented 1 year ago

@hjonnala any update when Python 3.10 compatible wheels will be released?

hjonnala commented 1 year ago

HI @rubenkr Unfortunately, we don't have any plans to release wheels for python 3.10. If possible please use 3.9. And with the current build scripts you won't be able to build for python 3.10. Please try making necessary changes to the below scripts. Thanks!!

https://github.com/google-coral/pycoral/blob/master/scripts/build.sh#L21 https://github.com/google-coral/pycoral/blob/master/scripts/build.sh#L45 https://github.com/google-coral/libcoral/blob/6589d0bb49c7fdbc4194ce178d06f8cdc7b5df60/docker/Dockerfile#L11

rubenkr commented 1 year ago

@hjonnala Thank you. I am using archlinux since months as my main server OS and feel uncomfortable to change this. As for pycoral should be supported in Linux I am waiting for other TPU coming to the market and returned my Coral device. Thank you!

google-coral-bot[bot] commented 1 year ago

Are you satisfied with the resolution of your issue? Yes No

Christian-Prather commented 1 year ago

@hjonnala Why is there no plans for Python 3.10 support? Id love to get this running on our embedded system (aarch64) but am restricted to Python 3.10. I've been trying to edit the files you listed above but keep running into XNNPACK memory undefined on build. Is there any support you could offer for pointing me in the write direction for building, I only ask because I saw the x86 version you have for 3.10. Thank you.

hjonnala commented 1 year ago

Hello @Christian-Prather please try the steps mentioned at : https://github.com/google-coral/pycoral/issues/85#issuecomment-1308139930, https://github.com/google-coral/pycoral/issues/85#issuecomment-1312166902 to build to locally..