search

coreos / fedora-coreos-tracker

Issue tracker for Fedora CoreOS
https://fedoraproject.org/coreos/
264 stars 59 forks source link

New Package Request: NetworkManager-wifi #862

Open dustymabe opened 3 years ago

dustymabe commented 3 years ago

I'm working on adding aarch64 support to our FCOS pipelines. Since I have a Raspberry Pi4 I'm also doing some enablement work to see what's needed to support it as a platform. Since it's a small device and can fit anywhere it's likely people could want to use wifi versus wired ethernet. We should consider adding the NetworkManager-wifi package to make this dead simple to do.

Please try to answer the following questions about the package you are requesting:

  1. What, if any, are the additional dependencies on the package? (i.e. does it pull in Python, Perl, etc)
Added:
  NetworkManager-wifi-1:1.30.4-1.fc34.x86_64
  iw-5.9-2.fc34.x86_64
  wireless-regdb-2020.11.20-2.fc34.noarch
  wpa_supplicant-1:2.9-12.fc34.x86_64
  1. What is the size of the package and its dependencies?
Will download: 4 packages (1.8 MB)
  1. What problem are you trying to solve with this package? Or what functionality does the package provide?

Wifi access.

  1. Can the software provided by the package be run from a container? Explain why or why not.

Not conveniently

  1. Can the tool(s) provided by the package be helpful in debugging container runtime issues?

Doubtful

  1. Can the tool(s) provided by the package be helpful in debugging networking issues?

Unlikely, unless your primary wired NIC is acting up.

  1. Is it possible to layer the package onto the base OS as a day 2 operation? Explain why or why not.

Yes. rpm-ostree install NetworkManager-wifi works.

  1. In the case of packages providing services and binaries, can the packaging be adjusted to just deliver binaries?

Not sure that would be useful. It's essentially a plugin for NetworkManager.

  1. Can the tool(s) provided by the package be used to do things we’d rather users not be able to do in FCOS? (e.g. can it be abused as a Turing complete interpreter?)

No.

  1. Does the software provided by the package have a history of CVEs?
bgilbert commented 3 years ago

Can the tool(s) provided by the package be used to do things we’d rather users not be able to do in FCOS? (e.g. can it be abused as a Turing complete interpreter?)

No.

I would qualify that a bit. People should probably not be running production container hosts on WiFi.

I do see the potential value to adding these packages for casual users and experimentation, but it's worth some thought about the cost/benefit ratio. For example, there are no defaults for WiFi connectivity, so users would always need to plumb connection info into the initramfs for Ignition. And supporting WiFi in the iniframfs might require some additional glue.

dustymabe commented 3 years ago

For example, there are no defaults for WiFi connectivity, so users would always need to plumb connection info into the initramfs for Ignition. And supporting WiFi in the iniframfs might require some additional glue.

Yeah, I haven't tried WiFi in the initramfs, but I was thinking people have two options there:

cgwalters commented 3 years ago

People should probably not be running production container hosts on WiFi.

You're thinking "datacenter or cloud" here right? One interesting thing I saw go by is: https://sites.cs.ucsb.edu/~ravenben/publications/abstracts/beam3d-hotnets11.html

BTW of course too, "production container host with WiFi" is a very IoT thing - small servers on manufacturing floor or placed on large infrastructure. And Fedora IoT does include both WiFi and cellular: https://pagure.io/fedora-iot/ostree/blob/main/f/fedora-iot-base.json#_87

So adding this would even further blur the lines between FCOS/IoT.

I don't have a really strong opinion on this myself. If we do this, we'll have to explain why our cloud images include WiFi support. But we'll also take a notable step towards eventual deduplication with IoT and other projects.

bgilbert commented 3 years ago

If we support WiFi at runtime, it'll be hard to explain why we don't support it at provisioning time. I don't think it'd take too long to start getting requests for it.

Re IoT, the FCOS PRD defines IoT as a use-case indifferent goal that isn't actively factored into design decisions. Since Fedora IoT already exists, I think it makes sense to let it handle the IoT cases and focus FCOS more on its own primary use cases.

dustymabe commented 3 years ago

I'm cool with the answer being a "no" on this, but I would like to run it through the meeting.

In the very least I think we should add some docs for "how do I get WiFi working on FCOS?"

travier commented 3 years ago

I think we should only provide networking support in the base image for the methods that can be used for provisioning. If the packages work fine when overlayed, then the case for installing them by default is weaker as this pushes them to every other platform by default where they are not needed.

This is a hard thing in FCOS: it's easy to add to the image, hard/risky to remove things.

But at the same time, the size bump is really small so not really a concern either (but it adds-up quickly as we add tools: kexec-tools, dnsmasq, etc.).

cgwalters commented 3 years ago

This intersects with https://github.com/coreos/fedora-coreos-tracker/issues/401 but in a particularly special way: anyone who wants this will need to do something like manually stick the required RPMs either inside their Ignition (eww) or we try to generalize support for embedding additional generic data into the ISO image which e.g. would be automatically copied into /boot on the target system. (That would also be useful for container images)

jlebon commented 3 years ago

This was discussed in today's meeting:

AGREED: we will document how to layer in WiFi packages for installations that need them

jlebon commented 2 years ago

This is related to https://github.com/coreos/rpm-ostree/pull/3006 and also https://github.com/coreos/fedora-coreos-tracker/issues/1151.

Anyway, here's a hacky way of how this could be done today:

# coreos-installer install /dev/vda ...
# mount /dev/vda4 /mnt/
# cd /mnt/ostree/deploy/fedora-coreos/var/tmp
# podman run --rm -v $PWD:/mnt --workdir /mnt --security-opt=label=disable \
    quay.io/fedora/fedora:35 sh -c \
        'dnf install -y dnf-plugins-core &&
         dnf download NetworkManager-wifi iw wireless-regdb wpa_supplicant'
# systemctl reboot
<reboot>

Then you can use a systemd unit similar to the one documented for adding OS extensions, except it would run rpm-ostree install --apply-live --cache-only --allow-inactive /var/tmp/*.rpm.

TBD whether we want that in the official FCOS documentation.

With https://github.com/coreos/rpm-ostree/pull/3538 we could make this slightly cleaner at least.

cgwalters commented 2 years ago

I also did https://github.com/coreos/coreos-layering-examples/pull/14 in response to this but forgot to cross-reference.

akostadinov commented 2 years ago

I've got it:

  1. create ignition
  2. install with that ignition
  3. before restart, modify files on installation target to include the RPMs
  4. restart only then

One issue with the above, because I downloaded more packages with the --resolve option, I've got the following error:

error: Package 'NetworkManager-1:1.32.12-2.fc35.x86_64' is already in the base

For whatever reason --allow-inactive and --idempotent do not help, see coreos/rpm-ostree#1837.

Two usability issues I see with this approach:

  1. installation must happen on wired connection (how do we also reinstall a remote machine?)
  2. there is no way to add files or ignition config to the installation USB media as it is using an iso format, that's to avoid network requirement by downloading files on some other machine

Would be nice if installation USB can have RPMs downloaded on it to be automatically picked-up by installer.

akostadinov commented 2 years ago

FYI I had success installing a few RPMs, getting network operational and then installing more RPMs online. But this solution is not optimal. The RPMs installed offline remain as local and I assume will not be upgraded with new versions online later on.

$ rpm-ostree status
State: idle
AutomaticUpdatesDriver: Zincati
  DriverState: active; periodically polling for updates (last checked Sun 2022-05-01 10:00:28 UTC)
Deployments:
● fedora:fedora/x86_64/coreos/stable
                   Version: 35.20220410.3.1 (2022-04-26T21:56:45Z)
                BaseCommit: 83cb95041c99fb812714d5c1544504d77dbb76b7798f0b6d59bf40356bd8d058
              GPGSignature: Valid signature by 787EA6AE1147EEE56C40B30CDB4639719867C58F
           LayeredPackages: ddclient iw vim vte-profile
             LocalPackages: NetworkManager-wifi-1:1.32.12-2.fc35.x86_64
                            wireless-regdb-2021.07.14-2.fc35.noarch
                            wpa_supplicant-1:2.10-2.fc35.x86_64

  fedora:fedora/x86_64/coreos/stable
                   Version: 35.20220327.3.0 (2022-04-11T16:21:43Z)
                BaseCommit: dcd18c80388d8b9266b52f2cba97d4f81b9f296f27e9505aa5f543efa422058c
              GPGSignature: Valid signature by 787EA6AE1147EEE56C40B30CDB4639719867C58F
           LayeredPackages: ddclient iw vim vte-profile
             LocalPackages: NetworkManager-wifi-1:1.32.12-2.fc35.x86_64
                            wireless-regdb-2021.07.14-2.fc35.noarch
                            wpa_supplicant-1:2.10-2.fc35.x86_64
dwarf-king-hreidmar commented 2 years ago

Can the tool(s) provided by the package be used to do things we’d rather users not be able to do in FCOS? (e.g. can it be abused as a Turing complete interpreter?) No.

I would qualify that a bit. People should probably not be running production container hosts on WiFi.

I do see the potential value to adding these packages for casual users and experimentation, but it's worth some thought about the cost/benefit ratio. For example, there are no defaults for WiFi connectivity, so users would always need to plumb connection info into the initramfs for Ignition. And supporting WiFi in the iniframfs might require some additional glue.

My cluster is using a switch for k8s cluster traffic but it uses wifi for egress/ingress. I may be using it on a small scale but I still consider it production. I could also foresee a use case for mobile clusters that support egress and ingress but otherwise are self contained.

akostadinov commented 2 years ago

To reduce final image size and to allow greater flexibility in deployment setup, it might be worth having pre-ceated package sets on the installation media that one can layer with ignition upon installation.

Or maybe make super easy to build coreos custom installation media.

dwarf-king-hreidmar commented 2 years ago

FYI I had success installing a few RPMs, getting network operational and then installing more RPMs online. But this solution is not optimal. The RPMs installed offline remain as local and I assume will not be upgraded with new versions online later on.

$ rpm-ostree status
State: idle
AutomaticUpdatesDriver: Zincati
  DriverState: active; periodically polling for updates (last checked Sun 2022-05-01 10:00:28 UTC)
Deployments:
● fedora:fedora/x86_64/coreos/stable
                   Version: 35.20220410.3.1 (2022-04-26T21:56:45Z)
                BaseCommit: 83cb95041c99fb812714d5c1544504d77dbb76b7798f0b6d59bf40356bd8d058
              GPGSignature: Valid signature by 787EA6AE1147EEE56C40B30CDB4639719867C58F
           LayeredPackages: ddclient iw vim vte-profile
             LocalPackages: NetworkManager-wifi-1:1.32.12-2.fc35.x86_64
                            wireless-regdb-2021.07.14-2.fc35.noarch
                            wpa_supplicant-1:2.10-2.fc35.x86_64

  fedora:fedora/x86_64/coreos/stable
                   Version: 35.20220327.3.0 (2022-04-11T16:21:43Z)
                BaseCommit: dcd18c80388d8b9266b52f2cba97d4f81b9f296f27e9505aa5f543efa422058c
              GPGSignature: Valid signature by 787EA6AE1147EEE56C40B30CDB4639719867C58F
           LayeredPackages: ddclient iw vim vte-profile
             LocalPackages: NetworkManager-wifi-1:1.32.12-2.fc35.x86_64
                            wireless-regdb-2021.07.14-2.fc35.noarch
                            wpa_supplicant-1:2.10-2.fc35.x86_64

Where is the best place to drop those rpm on the os image?

akostadinov commented 2 years ago

I used an external USB drive that I mounted sudo mount /dev/sdc1 /mnt. I don't know how to put them on the image. I think it will be easier just to generate an installation image with the packages layered and install, than doing what I did. But I have a problem with my server that I will return so will try when I get the new one.

I mean that I did a normal install with ISO written to USB flash. Then mount another USB with the packages to get network going. Then layer extra packages.

travier commented 2 years ago

FYI I had success installing a few RPMs, getting network operational and then installing more RPMs online. But this solution is not optimal. The RPMs installed offline remain as local and I assume will not be upgraded with new versions online later on.

Once online, you can do:

$ rpm-ostree remove NetworkManager-wifi-1:1.32.12-2.fc35.x86_64 ... --install NetworkManager-wifi ...

To use to the latest versions from the repos.

akostadinov commented 2 years ago

I don't have a really strong opinion on this myself. If we do this, we'll have to explain why our cloud images include WiFi support. But we'll also take a notable step towards eventual deduplication with IoT and other projects.

One size fits all does not make sense. container and cloud images make no sense to be same as bare-metal images. I recently started to use alpine for container images because resulting size is for example 15MB instead of 150 or more.

If there are a few images for common use cases and make it really easy to generate custom images, that will be liked by users.

In fact I'm soon trying to see how easy it is to generate a custom image. It may already be easy, I don't know. Just being stuck on one size fits all is not what the preferred approach seems to be in the community. Certainly not what I prefer.

akostadinov commented 2 years ago

FYI I had success installing a few RPMs, getting network operational and then installing more RPMs online. But this solution is not optimal. The RPMs installed offline remain as local and I assume will not be upgraded with new versions online later on.

Once online, you can do:

$ rpm-ostree remove NetworkManager-wifi-1:1.32.12-2.fc35.x86_64 ... --install NetworkManager-wifi ...

To use to the latest versions from the repos.

$ sudo rpm-ostree remove NetworkManager-wifi-1:1.32.12-2.fc35.x86_64 wireless-regdb-2021.07.14-2.fc35.noarch wpa_supplicant-1:2.10-2.fc35.x86_64 --install NetworkManager-wifi  wpa_supplicant wireless-regdb
ignoring ``: invalid filter directive
error: Package/capability 'NetworkManager-wifi' is already requested
bgilbert commented 2 years ago

cloud images make no sense to be same as bare-metal images.

The general idea is that your Fedora CoreOS node works the same whether you're running locally, in a datacenter on bare metal, or in the cloud.

dwarf-king-hreidmar commented 2 years ago

To reduce final image size and to allow greater flexibility in deployment setup, it might be worth having pre-ceated package sets on the installation media that one can layer with ignition upon installation.

Or maybe make super easy to build coreos custom installation media.

Instructions or an easy way to layer RPM to create your own coreos images would be amazing. Especially if those instructions include how to build the image for arm on a x86 machine.

travier commented 8 months ago

Given that we are looking at removing wifi firmwares from the image (https://github.com/coreos/fedora-coreos-tracker/issues/1575), I would say that we should close this issue.

We are working on path to make layering a more attractive option for this use case, including for the first boot.

dustymabe commented 8 months ago

I still think we'd want to document it, which was the last action item we decided on in https://github.com/coreos/fedora-coreos-tracker/issues/862#issuecomment-862678654

travier commented 8 months ago

Good point

dustymabe commented 7 months ago

work ongoing to add documentation in:

travier commented 7 months ago

https://github.com/coreos/fedora-coreos-docs/pull/629 has been merged, so I think we're good here now.