firmata / protocol

Documentation of the Firmata protocol.
987 stars 149 forks source link

Linux Kernel Drivers #141

Open TCB13 opened 1 year ago

TCB13 commented 1 year ago

Hello,

I've been using this for a while and it works great for simple GPIO communication however things get progressively harder with SPI or I2C. Although firmata does a very good job at implementing client libraries for multiple languages I noticed that there are zero references to a possible Linux driver.

A Linux driver would allow the GPIO, I2C and SPI devices to be exposed as linux devices in /dev/ via i2c-dev and spidev an improvement that would make everyone's life easier since there are tons of libraries for sensors and other devices that know how to communicate with SPI/I2C this way.

There are tons of FTDI chips (eg. FT232H) that implements things this way.

Thank you.

pgrawehr commented 1 year ago

I'm not sure I understand your request. Are you suggesting to write a kernel driver that routes requests to /dev/i2c (or similar) to an Arduino? While I do think that this might be helpful in some cases, I also think this is outside the scope of this project. We're providing the protocol specification and the firmware implementation, but providing such a client requires a different skill set.

There are client libraries available for all kinds of languages, can you tell me what language/system you're using that does not already have a high-level abstraction library?

TCB13 commented 1 year ago

Are you suggesting to write a kernel driver that routes requests to /dev/i2c (or similar) to an Arduino?

Yes, I'm suggesting that. With FTDI devices we also get that option. I'm suggesting this mostly because there tons of libraries for sensors, LCDs and other devices that come out of the box with support for linux / kernel devices and those can't be transparently used with an Arduino as bridge. Abstracting the lower lever communication into the OS by loading a driver decouples the program from the hardware...

Firmata is very good, and I also like the fact that there are language specific libraries but it could be way more flexible with a kernel driver, after all operating systems exist to abstract hardware into standard devices.

pgrawehr commented 1 year ago

I can see your point. However this is still a massive task that I'm currently unable to take. Also, the driver library I'm using and maintaining (https://github.com/dotnet/iot) does not have said problem, because it does allow transparently using all sensor drivers with all interfaces. Here's an example using a DHT11, a BMP280, a display and a button with Firmata.

TCB13 commented 1 year ago

I write this with the upmost respect for all the great work you've done, more like a joke: you're using .net so your perspective about "transparently using devices" might be a bit... skewed. :D

Now seriously, for example without a kernel driver any library that is smbus2 (I2C) or sysfs (GPIO) based wouldn't work:

Adafruit has Python libs for almost all common sensors out there and those tend to rely on Blinka (https://github.com/adafruit/Adafruit_Blinka). Blinka also supports generic I2C/GPIO/SPI using kernel devices (https://github.com/adafruit/Adafruit_Blinka/blob/6d7575069dd39a740212f7ae6c12ecf1c8f8bb98/src/adafruit_blinka/microcontroller/generic_linux/i2c.py).

Luma OLED, Luma LCD etc. (https://github.com/rm-hull/luma.oled) are also able to use Adafruit's Blinka devices to output images - libraries tend to pile up like this because it gives everyone a nice abstraction layer.

I might be mistaken but, for instance, I currently don't see a straightforward way to use firmata with Luma and a kernel driver would make it work out of the box without any further changes to those projects. Writing kernel drivers isn't an easy task but I was just trying to illustrate the benefits with real examples.

Alternatively a Firmata "client library" implemented for Adafruit's Blinka would be easier to make and still open the door for every other lib that relies on Blinka.

Thank you.

logicog commented 10 months ago

Hi TCB13, after reading this issue, and because I had the same problem, namely I needed a Linux kernel module (A CAN-bus driver for an MCP2515 which provides a network interface on Linux to the CAN bus) for a device connected to an Arduino connected to a PC host, I wrote: https://github.com/logicog/firmata_mod Could you give it a try? It should be exactly what you requested. I have tested it with GPIO and I2C as well as SPI devices and it works AFAICS. This is definitely alpha quality and will produce for example tons of kernel log entries, so nothing for production purposes! Still I believe this is something that could be at one point going into the Kernel, as it is really very useful for prototyping. However, it will need heavy testing and then cleanup. Please try!

TCB13 commented 10 months ago

Hi @logicog thank you very much for the efforts. This is truly remarkable and I'll test it as soon as get from my vacation. This opens a ton of options when it comes to controlling real world stuff from x86 machines.

Do you think we will be able to drive something an SPI display... something like an IPS RGB display such as an 800*480 NT35510, a smaller 320x240 IL19341 or a really small 128x128 SSD1351 at reasonable speeds?

pgrawehr commented 10 months ago

@TCB13 You would at least need to increase the serial port speed to something well above 1Mbit/s, which could however make the communication unstable. I am controlling an Ili9342 connected to an ESP32 over Wifi. The maximum troughput I'm getting is about 1MBit/s (probably limited by the ESP32), and this allows for about 2FPS. This isn't really good, but depending on the application, it can be enough. To improve this, some kind of data compression for the SPI data would be needed, I believe.

TCB13 commented 10 months ago

but depending on the application, it can be enough

Yes, a simple text based interface with a few menus for control would work fine at 2fps. I believe with an Arduino MEGA 2560 and Adafruit ILI9341 / Adafruit GFX I was able to drive that screen at much more than 2fps.

I'll what I can do with luma.lcd and report soon.

logicog commented 10 months ago

Hi, @TCB13, great that you want to give it a try!

I have not really tried running this at high throughput. Certainly moving the coding/decoding of the Firmata protocol into Kernel space will more likely than not increase performance compared to doing this in user space, and a 1MBit connection should be possible, all my uploads to an ESP32-C3 are at that speed and I have never seen issues.

So far I have only done tests at 115200 and 57600 baud. When testing at high speeds beware of the chattiness of the modules, they will literally spam your syslog with debug output at this point, every SPI r/w will produce multiple lines of debug printout. While this will probably not slow anything down given how powerful PCs are, your syslog will become rather difficult to use for any other purpose.

Good that you tell me you want to use this for SPI, I need to improve that part of the documentation ;-)