Preallocated observer_table can be limiting or wasteful

[snej]

The static observer_table preallocates ~1KB of RAM times NYOCI_MAX_OBSERVERS. So the client has to make a compile-time choice between supporting only a few observers — in embedded mode it defaults to 2 — or using a nontrivial amount of RAM for an embedded system. (My program will need to support quite a few observers, though the number will generally be small, so I left the max at 64; then I did some code-size profiling and saw that nyoci_observable.o is the single largest contributor to memory usage, at 77KB of .bss.)

This could be made dynamic by heap-allocating each nyoci_observer_s and making the table a heap-allocated growable array of pointers to the observers.

[darconeous]

Pre-allocating these sorts of resources is desirable behavior in many embedded systems. The case for pre-allocating resources is that the behavior of the device is much more predictable when resources run out. With pre-allocation, the availability of observer slots is independent from the allocations of other resources, and (more importantly) the allocation of other resources is independent of the number of observers (which could be many, especially if some are stale). Memory fragmentation can easily lead to inefficient memory usage.

Observation is complicated: making an implementation that gets all of the details right (such as respecting all of the options from the original request) is tricky—which is why I actually keep a copy of the original request, via a struct nyoci_async_response_s. This original request is then fed back as a "fake" request when the observable is triggered. That is the largest part of the observable object.

Note that we have a maximum size for an observable request, and that is dictated by NYOCI_ASYNC_RESPONSE_MAX_LENGTH, which does not include the 4 fixed bytes of the CoAP header. When NYOCI_EMBEDDED is true, it is set to a value of 80, otherwise it uses NYOCI_MAX_PACKET_LENGTH. So it only reserves ~1kb per observer if you aren't using NYOCI_EMBEDDED. NYOCI_ASYNC_RESPONSE_MAX_LENGTH can be specified independently, which will reduce your memory requirements by an order of magnitude.

Heap-allocating nyoci_observer_s might make sense on non-embedded platforms, but the current setup was hand-tuned to be space-efficient for embedded systems. For example, it uses a linked-list for associating observers, but instead of using pointers it uses a single index byte to keep track of the links. A lot of careful thought went into the correctness of that implementation, so changing it needs to be done carefully. It might make sense to have two implementations: one for embedded systems with a pre-allocated set of objects and one for non-embedded systems that allocates from the heap.

[darconeous]

Let me know if you continue to have memory problems after reducing the size of NYOCI_ASYNC_RESPONSE_MAX_LENGTH.