Replace userspace generated USB devices through ConfigFS with something more robust and portable

[doughsay]

Nerves out of the box uses a built-in kernel module CONFIG_USB_ETH=y. This is widely compatible across operating systems, but lacks the HID device we need for a keyboard.

The conventional wisdom to create composite USB devices is to use Linux's ConfigFS, which is what xebow does. (It creates a composite NCM Ethernet + RNDIS Ethernet + HID device, using ConfigFS: https://github.com/ElixirSeattle/xebow/blob/master/lib/xebow/hid_gadget.ex). The problem seems to be that we can't create an Ethernet device through ConfigFS that mimics the portability and robustness of the built-in kernel module... the NCM + RNDIS devices are a hack (one works on windows (RNDIS), the other works on linux and macOS (NCM)).

We need to either try harder to mimic the same behavior as the built-in module through configFS, or scrap configFS altogether, and maybe write our own composite HID + Ethernet kernel module based on the one Nerves uses out of the box.

I consider this problem "very hard".

[amclain]

@doughsay (cc @vanvoljg) do we want to reopen this issue? I'm not sure switching the Ethernet endpoint from ECM to NCM satisfied the intent of this issue (although it was a great step forward); my interpretation is that the ideal way to resolve this issue would be to expose a single Ethernet endpoint that works across all operating systems.

[doughsay]

Yes I think that might be wise. I mentioned in the ticket that I thought we should try writing our own kernel module, but that's hard, and exactly what configFS was built to solve.

On the other hand, I'm still unsure if there's a way to mimick the built in module with configFS, so that avenue should also be investigated.

What we've done is definitely a step in the right direction yes, but still not a desired end state.

[amclain]

Another thought I had in https://github.com/ElixirSeattle/xebow/pull/94#pullrequestreview-455047552 as a step towards this goal is to disable the Ethernet endpoint that is not able to initialize or obtain an IP address. For example, when the NCM endpoint establishes an IP on Linux, we disable the RNDIS endpoint so it stops attempting (and failing) to initialize. This could also make bonding the network adapters unnecessary. Hopefully we could detect the endpoint status by receiving notifications of the USB hotplug events and checking the network adapters at that time. If that doesn't work, maybe we could tap into the if up/down hooks. Worst-case maybe we poll the ifconfig of the adapters.

[vanvoljg]

I don't think there's a single USB networking standard that works over all three major operating systems. Hopefully, some day Windows will introduce a built in NCM driver that actually works.

One way I could think of that might help it work is for the device to know what type of host it's connected to, but from the logs I got yesterday, all the Keybow knows is, "new high speed device", so that would need more exploration.

What do we think of the idea of setting up a connection, then testing it from the device side? Can we do that? If it has no network connectivity, it would need to tear down the existing USB gadget to set up another one and test that. Is that a quick enough process that it would be minimally disruptive to the user?

It feels like there's going to end up being a USB disconnect and reconnect cycle, which I'm pretty sure would be noticeable (like flash of unstyled content kind of noticeable). I would feel weird with a device that connected, disconnected, and connected again. It would make me wonder if it was broken or buggy. (This very reason is why I don't like U3 flash drives.)

[amclain]

What do we think of the idea of setting up a connection, then testing it from the device side? Can we do that? If it has no network connectivity, it would need to tear down the existing USB gadget to set up another one and test that.

Yes, this is what I was saying. Except rather than test one at a time, test both at once and tear down the one that doesn't work (doesn't get an IP address).

Another option is to hold down a key on Xebow boot that forces it to RNDIS or NCM until the setting is changed. This is not necessarily mutually exclusive from the option above. For example, holding key 1 could enable auto detection, key 2 for RNDIS, and key 3 for NCM. We could mirror this setting in the web UI.

[doughsay]

No wait! Let's not give up!

I don't think there's a single USB networking standard that works over all three major operating systems.

I think this isn't true; see the statement in the issue:

Nerves out of the box uses a built-in kernel module CONFIG_USB_ETH=y. This is widely compatible across operating systems

I don't know what they do or how they do it but that kernel module DOES work on all OSes that I've tried. My first inclination is to understand how that module accomplishes it, and to attempt a composite gadget with whatever that is.