google-coral / edgetpu

Coral issue tracker (and legacy Edge TPU API source)
https://coral.ai
Apache License 2.0
411 stars 124 forks source link

Gasket fails on 6.5.x kernels #822

Open derekcentrico opened 4 months ago

derekcentrico commented 4 months ago

Coral is no longer available when following https://coral.ai/docs/m2/get-started/#2a-on-linux

Changes of some sort with kernel 6.5.x (all sub-version updates thus far from Ubuntu have problems).

Error: Setting up libedgetpu1-std:amd64 (16.0) ... Setting up gasket-dkms (1.0-18) ... Loading new gasket-1.0 DKMS files... Building for 6.5.0-17-generic Building initial module for 6.5.0-17-generic ERROR: Cannot create report: [Errno 17] File exists: '/var/crash/gasket-dkms.0.crash' Error! Bad return status for module build on kernel: 6.5.0-17-generic (x86_64) Consult /var/lib/dkms/gasket/1.0/build/make.log for more information. dpkg: error processing package gasket-dkms (--configure): installed gasket-dkms package post-installation script subprocess returned error exit status 10 Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Errors were encountered while processing: gasket-dkms E: Sub-process /usr/bin/dpkg returned an error code (1)

Log contains: DKMS make.log for gasket-1.0 for kernel 6.5.0-17-generic (x86_64) Wed Feb 14 09:59:02 AM EST 2024 make: Entering directory '/usr/src/linux-headers-6.5.0-17-generic' warning: the compiler differs from the one used to build the kernel The kernel was built by: x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0 You are using: gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0 CC [M] /var/lib/dkms/gasket/1.0/build/gasket_core.o CC [M] /var/lib/dkms/gasket/1.0/build/gasket_ioctl.o CC [M] /var/lib/dkms/gasket/1.0/build/gasket_interrupt.o CC [M] /var/lib/dkms/gasket/1.0/build/gasket_page_table.o CC [M] /var/lib/dkms/gasket/1.0/build/gasket_sysfs.o CC [M] /var/lib/dkms/gasket/1.0/build/apex_driver.o /var/lib/dkms/gasket/1.0/build/gasket_core.c: In function ‘gasket_register_device’: /var/lib/dkms/gasket/1.0/build/gasket_core.c:1841:41: error: passing argument 1 of ‘class_create’ from incompatible pointer type [-Werror=incompatible-pointer-types] 1841 class_create(driver_desc->module, driver_desc->name); ~~~^~~~
struct module *

In file included from ./include/linux/device.h:31, from ./include/linux/cdev.h:8, from /var/lib/dkms/gasket/1.0/build/gasket_core.h:11, from /var/lib/dkms/gasket/1.0/build/gasket_core.c:12: ./include/linux/device/class.h:230:54: note: expected ‘const char ’ but argument is of type ‘struct module ’ 230 | struct class __must_check class_create(const char name); | ~~^~ /var/lib/dkms/gasket/1.0/build/gasket_core.c:1841:17: error: too many arguments to function ‘class_create’ 1841 | class_create(driver_desc->module, driver_desc->name); | ^~~~ ./include/linux/device/class.h:230:29: note: declared here 230 | struct class __must_check class_create(const char name); | ^~~~ cc1: some warnings being treated as errors make[2]: [scripts/Makefile.build:251: /var/lib/dkms/gasket/1.0/build/gasket_core.o] Error 1 make[2]: Waiting for unfinished jobs.... make[1]: [/usr/src/linux-headers-6.5.0-17-generic/Makefile:2039: /var/lib/dkms/gasket/1.0/build] Error 2 make: [Makefile:234: __sub-make] Error 2 make: Leaving directory '/usr/src/linux-headers-6.5.0-17-generic'

bailboy91 commented 4 months ago

I am experiencing the same issues. kernel 6.5.x and 6.6.x.

derekcentrico commented 4 months ago

Apparently, there's a way to build the dependences ourselves. But, I don't think that's the best route. Their PPA should maintain updated debs.

bailboy91 commented 4 months ago

I built gasket-dkms from here https://github.com/google/gasket-driver/ on kernel 6.6.17 just now with no issues.

I do agree though. Their PPA should have an updated package for certain.

JS1Dev commented 1 month ago

Getting the same error following the same https://coral.ai/docs/m2/get-started/#2a-on-linux

The kernel was built by: x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
  You are using:           gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0

@bailboy91 - Is https://github.com/google/gasket-driver where you build the gasket-dkms from? And if so, would you mind giving some steps you used to do it?

bailboy91 commented 1 month ago

@JS1Dev Luckily I think I have enough in my bash history to get you at the very least in the right direction.
Here are the steps below that I took:


git clone https://github.com/google/gasket-driver.git
cd gasket-driver/
debuild
sudo apt-get install devscripts build-essential lintian debhelper 
debuild -us -uc -tc -b
cd ..
sudo dpkg -i gasket-dkms_1.0-18_all.deb

I re ran through those commands and installed the latest again just to confirm it was working. @ me again if you need any help! I built this on 22.04 with kernel 6.6.29. And it's been working through multiple kernel updates without issue for me.

JS1Dev commented 1 month ago

Hi @bailboy91, VERY nice of you to take the time to write down the steps to help me an others who might stumble upon this until it's eventually fixed. Unfortunately I got tripped up at the step to run debuild where it said that it's not found but could be installed using sudo apt install devscripts, which circled me back to the same issue as follows

$ debuild
Command 'debuild' not found, but can be installed with:
sudo apt install devscripts
$ sudo apt install devscripts

(^^^) produced the following error:

ERROR: Cannot create report: [Errno 17] File exists: '/var/crash/gasket-dkms.0.crash'
Error! Bad return status for module build on kernel: 6.5.0-28-generic (x86_64)
Consult /var/lib/dkms/gasket/1.0/build/make.log for more information.
dpkg: error processing package gasket-dkms (--configure):
 installed gasket-dkms package post-installation script subprocess returned error exit status 10
Errors were encountered while processing:
 gasket-dkms
E: Sub-process /usr/bin/dpkg returned an error code (1)

/var/lib/dkms/gasket/1.0/build/make.log brings me back to the same error

DKMS make.log for gasket-1.0 for kernel 6.5.0-28-generic (x86_64)
Mon May 13 09:28:21 AM EDT 2024
make: Entering directory '/usr/src/linux-headers-6.5.0-28-generic'
warning: the compiler differs from the one used to build the kernel
  The kernel was built by: x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
  You are using:           gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
  CC [M]  /var/lib/dkms/gasket/1.0/build/gasket_core.o
  CC [M]  /var/lib/dkms/gasket/1.0/build/gasket_ioctl.o
  CC [M]  /var/lib/dkms/gasket/1.0/build/gasket_interrupt.o
  CC [M]  /var/lib/dkms/gasket/1.0/build/gasket_page_table.o
  CC [M]  /var/lib/dkms/gasket/1.0/build/gasket_sysfs.o
/var/lib/dkms/gasket/1.0/build/gasket_core.c: In function ‘gasket_register_device’:
/var/lib/dkms/gasket/1.0/build/gasket_core.c:1841:41: error: passing argument 1 of ‘class_create’ from incompatible pointer type [-Werror=incompatible-pointer-types]
 1841 |                 class_create(driver_desc->module, driver_desc->name);
      |                              ~~~~~~~~~~~^~~~~~~~
      |                                         |
      |                                         struct module *
In file included from ./include/linux/device.h:31,
                 from ./include/linux/cdev.h:8,
                 from /var/lib/dkms/gasket/1.0/build/gasket_core.h:11,
                 from /var/lib/dkms/gasket/1.0/build/gasket_core.c:12:
./include/linux/device/class.h:230:54: note: expected ‘const char *’ but argument is of type ‘struct module *’
  230 | struct class * __must_check class_create(const char *name);
      |                                          ~~~~~~~~~~~~^~~~
/var/lib/dkms/gasket/1.0/build/gasket_core.c:1841:17: error: too many arguments to function ‘class_create’
 1841 |                 class_create(driver_desc->module, driver_desc->name);
      |                 ^~~~~~~~~~~~
./include/linux/device/class.h:230:29: note: declared here
  230 | struct class * __must_check class_create(const char *name);
      |                             ^~~~~~~~~~~~
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:251: /var/lib/dkms/gasket/1.0/build/gasket_core.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [/usr/src/linux-headers-6.5.0-28-generic/Makefile:2039: /var/lib/dkms/gasket/1.0/build] Error 2
make: *** [Makefile:234: __sub-make] Error 2
make: Leaving directory '/usr/src/linux-headers-6.5.0-28-generic'
bailboy91 commented 1 month ago

Oh no! I left in my debuild without flags and didn't notice. Can you try it with like this: debuild -us -uc -tc -b

JS1Dev commented 1 month ago

Thanks @bailboy91, but still no luck, debuild -us -uc -tc -b produced this:

$ debuild -us -uc -tc -b
 dpkg-buildpackage -us -uc -ui -tc -b
dpkg-buildpackage: info: source package gasket-dkms
dpkg-buildpackage: info: source version 1.0-18
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Coral <coral-support@google.com>
 dpkg-source --before-build .
dpkg-buildpackage: info: host architecture amd64
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper (>= 9)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui -tc -b failed

So I tried installing the dependency with sudo apt-get install debhelper but that brought me back to the same error from the original post.

For tracking purposes these are the steps so far

git clone https://github.com/google/gasket-driver.git
cd gasket-driver/
# before the prior post (^^^) below was just: `debuild` but with the advised changes (below), I get `error: Unmet build dependencies: debhelper`
debuild -us -uc -tc -b
# To resolve `error: Unmet build dependencies: debhelper` from (^^^), I install that dependency as follows, which results in the same original post error again:
`sudo apt-get install debhelper`

# I dont get this far
sudo apt-get install devscripts build-essential lintian debhelper 
debuild -us -uc -tc -b
cd ..
sudo dpkg -i gasket-dkms_1.0-18_all.deb
bailboy91 commented 1 month ago

sudo apt-get install devscripts build-essential lintian debhelper

Make sure these are installed before you try debuild.

JS1Dev commented 1 month ago

Unfortunately, the following error (same as the original post) must relate to everything you're saying needs to be installed so I'm not able to have the success you had:

The kernel was built by: x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
  You are using:           gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0

sudo apt-get install devscripts results in:

Error! Bad return status for module build on kernel: 6.5.0-28-generic (x86_64)
Consult /var/lib/dkms/gasket/1.0/build/make.log for more information.

where /var/lib/dkms/gasket/1.0/build/make.log shows the same error as in this post

Per my last post, sudo apt-get install debhelper resulted in the same /var/lib/dkms/gasket/1.0/build/make.log

bailboy91 commented 1 month ago

OH! I wonder if apt is in a weird failed state from attempting to setup the dkms module before custom building it from source. What does sudo apt install -f do?

if all else fails try sudo dpkg --remove gasket-dkms then try installing those dependencies for debhelper ect.

JS1Dev commented 1 month ago

Success!!! Quick summary

QUESTION

Detailed step of how I got there (for others that this may benefit): $ sudo apt install -f produced the same error as the original post in: resulted in the same /var/lib/dkms/gasket/1.0/build/make.log

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Setting up gasket-dkms (1.0-18) ...
Removing old gasket-1.0 DKMS files...
Deleting module gasket-1.0 completely from the DKMS tree.
Loading new gasket-1.0 DKMS files...
Building for 6.5.0-28-generic
Building initial module for 6.5.0-28-generic
ERROR: Cannot create report: [Errno 17] File exists: '/var/crash/gasket-dkms.0.crash'
Error! Bad return status for module build on kernel: 6.5.0-28-generic (x86_64)
Consult /var/lib/dkms/gasket/1.0/build/make.log for more information.
dpkg: error processing package gasket-dkms (--configure):
 installed gasket-dkms package post-installation script subprocess returned error exit status 10
Errors were encountered while processing:
 gasket-dkms
E: Sub-process /usr/bin/dpkg returned an error code (1)

sudo dpkg --remove gasket-dkms successfully uninstalled dpkg sudo apt-get install devscripts build-essential lintian debhelper was "successful" (see below)

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
build-essential is already the newest version (12.9ubuntu3).
build-essential set to manually installed.
debhelper is already the newest version (13.6ubuntu1).
devscripts is already the newest version (2.22.1ubuntu1).
lintian is already the newest version (2.114.0ubuntu1.3).
lintian set to manually installed.
The following package was automatically installed and is no longer required:
  dkms
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

per (^^^) I ran sudo apt autoremove which successfully removed dkms I then ran debuild -us -uc -tc -b but it failed per build dependencies/conflicts unsatisfied

 dpkg-buildpackage -us -uc -ui -tc -b
dpkg-buildpackage: info: source package gasket-dkms
dpkg-buildpackage: info: source version 1.0-18
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Coral <coral-support@google.com>
 dpkg-source --before-build .
dpkg-buildpackage: info: host architecture amd64
dpkg-checkbuilddeps: error: Unmet build dependencies: dkms
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui -tc -b failed

I then did sudo apt update (7 packages can be upgraded) then sudo apt upgrade, which all completed successfully I then tried debuild -us -uc -tc -b again, but it failed per Unmet build dependencies: dkms I then installed sudo apt-get install dkms (which was successful) Then debuild -us -uc -tc -b was finally successful! :) I then went back to do your final steps (cd .. and sudo dpkg -i gasket-dkms_1.0-18_all.deb), which completed successfully (ending in the following)

Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/6.5.0-35-generic/updates/dkms/

depmod...

Backing up initrd.img-6.5.0-35-generic to /boot/initrd.img-6.5.0-35-generic.old-dkms
Making new initrd.img-6.5.0-35-generic
(If next boot fails, revert to initrd.img-6.5.0-35-generic.old-dkms image)
update-initramfs.......

I then rebooted and no signs of an issue (though I'm not sure where to look to verify)

bailboy91 commented 1 month ago

Excellent! I use a project called frigate with my pcie coral. I've not tested with pytorch or anything else though. so long as you see an apex device in /dev/ you should be all set.

JS1Dev commented 1 month ago

Me too :) As far as I can tell this looks like it's working, thanks so much for your help! image image

kevin-griffiths-btinternet-com commented 5 days ago

Thank you both. I have been going round in circles for two days trying to follow the instructions on https://coral.ai/docs/m2/get-started/#2b-on-windows. I was looking for a purge command and found this by accident. Thanks again KevinG