Fluent Bit for Microcontrollers

[beriberikix]

Has the project considered supporting tiny embedded microcontrollers? The two main challenges I see are RAM usage and OS dependency.

At 450KB, that's more than most embedded systems have for the entire device. Typical devices may have 128K~256K for the OS/runtime, networking and user application. Also, most embedded systems don't have a traditional operating system. They typically have a highly-optimized library called an RTOS. Anything in Fluent Bit that assumes Linux (from stdlib to mmu to networking) won't work or need to be replaced with a smaller variant.

So how might Fluent Bit be able to target embedded system? I can imagine setting low memory limits removing any OS dependencies, using more efficient libraries like newlib and probably excluding certain features that just can't work in these types of environments.

Is that feasible and something that has been considered?

[edsiper]

Has the project considered supporting tiny embedded microcontrollers?

Yes

The question has always been: what's the desired feature set for an embedded platform ?

I think if the goal is to have a simple lib with basic API to ingest records in a few end-points we could write something from scratch

[beriberikix]

I think that would be useful. In particular, having a standardized Library API. For example, different MCU vendors have their own flavor of logging frameworks (ex1 ex2) which could be implemented as inputs and more constrained-friendly output protocols (like MQTT and CoAP.)

[nigels-com]

We had some success using fluent-bit with uClibc on embedded ARM. The three lowest hanging fruit I came across if I recall correctly:

1. FLB_SMALL build mode (-Os) https://github.com/fluent/fluent-bit/pull/1459
2. Disable TLS, if you can do without the security. We were considering linking to shared-lib curl instead, which is already required on the firmware for other reasons.
3. Disable regex support, it's quite large too.

[beriberikix]

@edsiper watching your cloudnativesummit.online reminded about this discussion! Have you looked into supporting the embedded MCU use case?

[edsiper]

@beriberikix thanks for participating :)

since we have our own event loop tied to an OS implementation, I am not confident it will be good for a MCU embedded use case. But I am pretty open to implement any input plugin that can help to connect those devices.

Anyways if stripping down even more Fluent Bit works, happy to help.

[beriberikix]

How tightly coupled is the event loop tied to the OS? Any pointers that might help me understand how it is implemented?

The reason I ask is because many embedded applications use a framework, aka RTOS, that provide OS primitives like threads, mutexes, semaphores, etc. Depending how the event loop is implemented, it may be possible to create thin layers that map to the popular implementations, like mbed, Zephyr & FreeRTOS.

[edsiper]

on Linux we use epoll(7) and MacOS & BSD we use kqueue.

but for an embedded device like that, which specific features from Fluent BIt are you interested in ? (I want to understand the logging requirements for a such small device)

[beriberikix]

I've been thinking more about this lately and I'm still on the fence if it makes sense for Fluent Bit. However, here's the types of things that one might log on an embedded/IoT device in either a healthy or debug mode:

• RTOS info (Thread start, network interface up, build id, etc.)
• System Error (Coredumps, backtraces, variables, registers, etc.)
• Networking metadata (IP addr, raw packets last ping, MAC, subnet, cell tower RSSI, carrier tracing info, etc.)
• User application logging (printf-stuff, hardware peripheral state)

So the question to answer is are those the kinds of things one might use Fluent Bit for and does it make to adapt Fluent Bit? Most likely some of that might be store in non-volatile memory but most likely it would be sent over the air due to resource constraints on the device.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.