[Task / Story]: Transition wifi drivers to DKMS modules

ColorfulRhino commented 4 months ago

Task description

The current status

Currently, 3rd-party wifi drivers cannot be installed on a per-board basis. Although some drivers are restricted to a specific LINUXFAMILY, many drivers are installed across all kernels. This approach leads to challenges because these drivers often require numerous patches (as shown in drivers_network.sh). These patches will frequently fail on new kernel versions and need to be carefully repaired, or new patches have to be crafted/imported. This issue becomes particularly pronounced when updating to -rc kernel releases, such as those recently seen with the rk3588 family, where drivers may not yet be compatible with the latest kernel version.

Consequently, those who spend their time updating kernels to newer versions face an additional burden of fixing all the 3rd-party wifi drivers that might not be updated by their original maintainers This ultimately means that the people who do lots of work to update kernels to newer versions are having this additional hurdle to jump over, to try and fix all the 3rd-party wifi drivers that may or may not be updated to the latest kernel version by their creators.

Proposed solution

Fix this by creating DKMS packages for the drivers which can then be installed on a per-board basis for boards with specific wifi chips or on-demand via the Armbian repo. This approach offers several benefits:

Board maintainers can directly fix and test the wifi driver for their board.
Maintainers can see changes immediately and have a stronger incentive to compare the driver with the one potentially included in the mainline kernel.
From a user's perspective, some users may not want to have 3rd-party kernel modules installed the same way they don't want to trust vendor kernels. This approach won't force all those 3rd-party wifi drivers to be installed the user's device together with the kernel (additional security benefits from a user's perspective).

Task list

[ ] Figure out the best design: Should we package the dkms modules and upload them to the armbian repo to be simply added as package at build time? Or shoudl we use extentions like in this example: https://github.com/armbian/build/blob/main/extensions/radxa-aic8800.sh
[ ] Develop a DKMS package creation mechanism
[ ] Transition all wifi drivers from drivers_network.sh to the new DKMS mechanism
[ ] Enable/install the DKMS packages preinstalled per board that uses the specific wifi chip, in the board config file (either with an extension, a variable like DKMS_MODULE= or with some other config hook)

NB: Anyone is welcome to take up this task and contribute to this description/task list :) I do not plan to do this all on my own.

github-actions[bot] commented 4 months ago

Jira ticket: AR-2416

igorpecovnik commented 4 months ago

Board maintainers can directly fix and test the wifi driver for their board. This ultimately means that the people who do lots of work to update kernels to newer versions are having this additional hurdle to jump over, to try and fix all the 3rd-party wifi drivers

In many cases this is done by 3rd party projects such as https://github.com/morrownr/USB-WiFi We bump commit ids and maintain a few of that are not covered by anyone else. For those few and a few end users that will get the same code in a different package?

From a user's perspective, some users may not want to have 3rd-party kernel modules installed the same way they don't want to trust vendor kernels.

Most users wants that WiFi works when they plug it in ... and on this idea this part of the Armbian is based upon. Practical way. Such ideas will always clash with developers perspectives. But at the end of the day, nobody really wants to compile drivers.

If this is a serious problem, perhaps adding a disclaimers on download pages / welcome text? Most people are not aware of this. Also for vendor kernel are not clear what it is. For you and me, yes, for average Joe, no.

We discussed option to drop those drivers in the past. Due to maintainace costs vs. real usability and actual needs. Common drivers should not be done by Armbian. We don't have capacity nor help from any WiFI vendor nor Linux foundation or similar organisation. This is where my fear lies. Wireless networking is serious problem / Pandora box.

DKMS mechanism

Is this realistic to have this and keep it there on all default by user-space compilers? I am afraid not with all as upstream struggle already. Some we are dealing with in as-cheap-as-possible HW are broken beyond reasonable repair. This sounds like more work then now and providing something on higher levels. Without counting "getting there" part.

I don't want to sound discouraging, or the one that wants to see the dark sides :) I need to see better rationale.

Transition all wifi drivers from

All those drivers should be tested after this. We need to be sure they are at least not working worse. Which is again more work for the team and something that require serious coordination to extract usable info and more to use that info. I tried to workaround this by building automated test station, but project is at hold for almost two years (as its yet again not that simple as it sounds) now and as there are so many other problems that also needs to be resolved ... If we go into any of this direction, lack of usable test automation is serious problem for R&D. Most of users will also never know we are doing anything (if there is no promotion about) and they will anyway report problems on various forums, Facebooks, 3rd party projects ... how to collect all this to know where you are and who will fix broken DKMS after? Which is new problem in this armbian-low-end-wireless drivers picture. Is it smaller or bigger then what it is now?

ColorfulRhino commented 4 months ago

Thanks for your reply!

In many cases this is done by 3rd party projects such as https://github.com/morrownr/USB-WiFi We bump commit ids and maintain a few of that are not covered by anyone else.

Yes, that's a good thing that this exists at all. In theory, that doesn't sound "too hard to maintain". I can only talk from my own experience, I have enjoyed trying to bring boards to newer kernels. But sometimes the wifi drivers were a roadblock that stopped me from progressing for the actual kernel and board I was working on, because driver xy could not compile and so on. Not always, but when it did, it really sucked the joy out of working on that for me. I tried to make it a little bit easier to patch the drivers by adding URLs so you can just click on the link and are directly in the repo, and also added dates to the commits so you can compare the date of the commits instead of having to compare SHA hashes which aren't really meant to be compared by a human. But ultimately, this did not make the big difference I had hoped for.

I'm sure that there are some maintainers who enjoy working on wifi drivers, so I'd like to leave this to them :)

Most users wants that WiFi works when they plug it in ... But at the end of the day, nobody really wants to compile drivers.

Unfortunately I have to agree to this statement.

If this is a serious problem, perhaps adding a disclaimers on download pages / welcome text? Most people are not aware of this. Also for vendor kernel are not clear what it is. For you and me, yes, for average Joe, no.

Raising awareness is always a good idea 👍 Although to say "if you don't want this drivers in your kernel, you'll have to build an image yourself and you won't receive any updates from Armbian" might be quite discuraging for users 😅

We discussed option to drop those drivers in the past. Due to maintainace costs vs. real usability and actual needs. Common drivers should not be done by Armbian. We don't have capacity nor help from any WiFI vendor nor Linux foundation or similar organisation. This is where my fear lies. Wireless networking is serious problem / Pandora box.

Yes, it is a similar pandora box like with the tv boxes. Maybe we should start by adding some recommended wifi adaptors to the website download section where people can see it? We can recommend actually good wifi adaptors (USB, M.2, ...) that are supported by mainline, where we don't have to do any maintenance. Users are happy because they have a good adaptor that just works and Armbian is happy because less issues 😄 What does everyone think about this idea?

Some we are dealing with in as-cheap-as-possible HW are broken beyond reasonable repair.

Yeah, this is of course the other problem, vendors putting cheap unsupported wifi chipsets on some boards. Personally I wouldn't even want to be a maintainer for such a board. But some people would want to support this and that's fine.

This sounds like more work then now and providing something on higher levels. Without counting "getting there" part. ... Which is new problem in this armbian-low-end-wireless drivers picture. Is it smaller or bigger then what it is now?

I don't want it to be more work or be more complex than it currently is, that seems like a step backwards 😅 The solution with DKMS is just one idea. I simply wanted to document this and propose one solution. This is just a ticket and anyone is welcome to discuss this topic 😄 Maybe someone else has a better idea :)

as its yet again not that simple as it sounds

Ahhh, it's never as simple as it sounds, I know 😅 But automation in general is the way :) More work in the beginning, but then way less work in the future (if done right). (Doesn't apply for everything of course)

rpardini commented 4 months ago

Few points here:

1) we really only have out-of-tree Wifi drivers working at all because we force the family maintainers to fix them when bumping. as a family maintainer, I hate it; as an user of for example uwe stuff, I love it. I really can't make up my mind about it. I personally take the stance "try to fix, if way too much work, disable that specific driver for that family".

2) mainline linux has been adding rtw stuff in-tree. managing the migration from out-of-tree to mainline though is a huge, separate chore, as seen in many issues/prs before this. some chipsets will perform worse in mainline initially or will require specific firmware. some vendors rely on Armbian having the out-of-tree drivers, so simply moving all to mainline won't work, at least in the foreseeable future...

2) drivers and firmware go hand in hand. We need to re-architect the firmware packaging. today we only have either "small" or "full" firmware. even the small is already huge, and out-of-tree-driver-supporting-firmware is a large part of it. Having per-family (and/or, per-driver?!) firmware might be more pressing.

3) I've a feeling most of the "code" doing our wifi drivers are actually "un-doing" the DKMS'ing of the upstream.

4) We end up with a huge "patch" being applied to every kernel, as the drivers are all cobbled together in a single commit/patch, due to the fact multiple drivers "patch" (with sed!) the same Makefile, at exactly the same spot, and conflicts would arise otherwise.

5) I think this might be only for uwe stuff, but in that case, Armbian is the upstream for that driver, even if it was cobbled together from elsewhere. The fact we have both sunxi and rockchip boards with it, and thus, different-per-family driver patching is just 🤯

arkadijs commented 4 months ago

Maybe we should start by adding some recommended wifi adaptors to the website download section where people can see it?

FTR, there is already a list: https://github.com/morrownr/USB-WiFi/blob/main/home/USB_WiFi_Adapters_that_are_supported_with_Linux_in-kernel_drivers.md#axe3000---usb30---24-ghz-5-ghz-and-6-ghz-wifi-6e Mediatek mt7921au: Fenvi AX1800 for 12,- Eur on AliExpress - best WiFi dongle ever since v5.18. A little large, may block adjacent ports but cables / USB-C to USB-A hubs can be had for additional 3,- Eur.

armbian / build