search

hypriot / image-builder-rpi

SD card image for Raspberry Pi with Docker: HypriotOS
http://blog.hypriot.com/post/how-to-get-docker-working-on-your-favourite-arm-board-with-hypriotos/
MIT License
1.07k stars 168 forks source link

linux-headers includes x86_64 tools... #215

Closed brendandburns closed 6 years ago

brendandburns commented 6 years ago

Trying to compile a kernel module with DKMS fails because tools like scripts/recordmcount are compiled for x86_64 not ARM...

This is due to cross-compiling on x86. We should probably build the package on rpi so that these tools are also rpi compatible.

For now you can re-compile them individually as a workaround...

StefanScherer commented 6 years ago

Thanks @brendandburns for "flying" HypriotOS :-)

I tried to reproduce the problem by installing the kernel headers we also provide as a package. I flashed the latest HypriotOS 1.7.1 and did the following steps:

$ sudo apt-get update
$ sudo apt-get install raspberrypi-kernel-headers
$ /usr/src/linux-headers-4.4.50-v7+/scripts/recordmcount
usage: recordmcount [-w] file.o...

The tool shows me the usage, so I think this is compiled for ARM. Maybe you have installed the linux headers from a different package?

Our RPI kernel is built in repo https://github.com/hypriot/rpi-kernel and the deb packages are deployed to https://packagecloud.io/Hypriot/rpi

So if a kernel module is missing, maybe we should add this to our pipeline?

brendandburns commented 6 years ago

I will try this again this evening... I'm trying to build the iscsitarget module... Not sure if we want to include that or not...

--brendan

From: Stefan Scherer notifications@github.com Sent: Monday, February 19, 2018 5:10:29 AM To: hypriot/image-builder-rpi Cc: Brendan Burns; Mention Subject: Re: [hypriot/image-builder-rpi] linux-headers includes x86_64 tools... (#215)

Thanks @brendandburnshttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbrendandburns&data=04%7C01%7Cbburns%40microsoft.com%7C570781c7e83c4f68dbe008d5779a26f9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636546426320207219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=Bvn9GaD4lD%2FOCRbl2ljgkCCmHJFZivZLXPdujGEfZBE%3D&reserved=0 for "flying" HypriotOS :-)

I tried to reproduce the problem by installing the kernel headers we also provide as a package. I flashed the latest HypriotOS 1.7.1 and did the following steps:

$ sudo apt-get update $ sudo apt-get install raspberrypi-kernel-headers $ /usr/src/linux-headers-4.4.50-v7+/scripts/recordmcount usage: recordmcount [-w] file.o...

The tool shows me the usage, so I think this is compiled for ARM. Maybe you have installed the linux headers from a different package?

Our RPI kernel is built in repo https://github.com/hypriot/rpi-kernelhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fhypriot%2Frpi-kernel&data=04%7C01%7Cbburns%40microsoft.com%7C570781c7e83c4f68dbe008d5779a26f9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636546426320207219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=ED2XZ8xR3xep5Kn7%2Bw3Pwhdg9eFTZJ41hZ09yyXNDeg%3D&reserved=0 and the deb packages are deployed to https://packagecloud.io/Hypriot/rpihttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackagecloud.io%2FHypriot%2Frpi&data=04%7C01%7Cbburns%40microsoft.com%7C570781c7e83c4f68dbe008d5779a26f9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636546426320217233%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=ZI98KxAuOgyTq9PYbOWnmFPmoWWlayrB3M1Hh2OYmHU%3D&reserved=0

So if a kernel module is missing, maybe we should add this to our pipeline?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fhypriot%2Fimage-builder-rpi%2Fissues%2F215%23issuecomment-366688897&data=04%7C01%7Cbburns%40microsoft.com%7C570781c7e83c4f68dbe008d5779a26f9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636546426320217233%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=JjIoidfBjGUM%2FBzk7hTW905F4My%2FFtGgDyRy%2BggtqZw%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFfDgkkSlOkWb3Rl76eGVIJiMY7XXxRxks5tWXLFgaJpZM4SJ1un&data=04%7C01%7Cbburns%40microsoft.com%7C570781c7e83c4f68dbe008d5779a26f9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636546426320217233%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=NOCKMbvBSIOml%2FIXfu8ZiY5eH0f13Emokwni809c0Hc%3D&reserved=0.

brendandburns commented 6 years ago

To use DKMS, I believe I need the following installed:

linux-headers-4.4.50-hypriotos-v7+_4.4.50-hypriotos-v7+-1_armhf.deb

https://packagecloud.io/Hypriot/rpi/packages/debian/jessie/linux-headers-4.4.50-hypriotos-v7+_4.4.50-hypriotos-v7+-1_armhf.deb

which has the x86_64 tools inside it...

Many thanks for any help here...

brendandburns commented 6 years ago

@StefanScherer

friendly ping on this if there is any way for me to fix/recompile this package for arm...

Thanks! --brendan

StefanScherer commented 6 years ago

Thanks @brendandburns for the ping. I was down with influenza short after my last reply and also had to cancel my trip to MVP Summit this week. Otherwise we had the chance to hack on this in Redmond ;-)

What I don't understand is, I can't see any x86_64 tools inside this package. As mentioned above I've installed the package (with sudo apt-get update && sudo apt-get install raspberrypi-kernel-headers) on a Raspberry Pi 3 with HypriotOS 1.7.1 installed. When I check all tools that are in the scripts folders

$ cd /usr/src/linux-headers-4.4.50+/scripts
$ file * | grep ELF
conmakehash:                       ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=2c3a1e539429dd1d4b238562decb326faeb06e2c, not stripped
kallsyms:                          ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=94af920e5874f37a9d745ca4ba063dd06dc5b832, not stripped
pnmtologo:                         ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=31b9263651369b4f570f3a1209b9c480ec4ee424, not stripped
recordmcount:                      ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=2a87ecdd01da5e542f3e55d679e401c3d9a2a5f9, not stripped
sortextable:                       ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=b28c91f1371f766790d2f6cd69b3ad4ea2f83a3f, not stripped
HypriotOS/armv7: pirate@black-pearl in /usr/src/linux-headers-4.4.50+/scripts
$ cd ../..
HypriotOS/armv7: pirate@black-pearl in /usr/src
$ cd linux-headers-4.4.50-v7+/scripts/
HypriotOS/armv7: pirate@black-pearl in /usr/src/linux-headers-4.4.50-v7+/scripts
$ file * | grep ELF
conmakehash:                       ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=2c3a1e539429dd1d4b238562decb326faeb06e2c, not stripped
kallsyms:                          ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=94af920e5874f37a9d745ca4ba063dd06dc5b832, not stripped
pnmtologo:                         ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=31b9263651369b4f570f3a1209b9c480ec4ee424, not stripped
recordmcount:                      ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=2a87ecdd01da5e542f3e55d679e401c3d9a2a5f9, not stripped
sortextable:                       ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=b28c91f1371f766790d2f6cd69b3ad4ea2f83a3f, not stripped

every ELF binary is compiled for ARM.

Do you have any command I should check in my environment to see the error you have?

To compile this package I think the most lazy way would be to fork https://github.com/hypriot/rpi-kernel and activate the CircleCI build in that fork. Then you can adjust the two files

Circle is configured to store all deb files as artifacts, see our last build 59 in master branch and the corresponding artifacts:

bildschirmfoto 2018-03-10 um 22 50 56

So you are able to download the deb packages to your Pi and let a cloud server do the dirty task of compiling.

The compilation is done in the scripts/compile_kernel.sh, of course cross compiled on an x86_64 machine on Circle.

I haven't compiled the kernel natively on a Raspberry Pi for years, the Raspberry Pi docs have a page Kernel Building.

brendandburns commented 6 years ago

Ok, with the latest headers I don't see this. I don't know what the issue was...

But sadly I'm not building compatible modules still, I will send a repro in a different issue.