search

njaard / sonnerie

A simple timeseries database
Other
268 stars 19 forks source link

Feature Request: Support for 16-bit and Arbitrary-Sized Integers and Floats #43

Open TKaluza opened 1 week ago

TKaluza commented 1 week ago

Currently, Sonnerie supports storing series in row format using specific formats defined by single-character codes. These formats are limited to:

However, it would be highly beneficial to add support for additional types, such as:

  1. 16-bit integers (signed and unsigned)
  2. 16-bit floats
  3. Arbitrary-sized integers and floats (e.g., 8-bit, 10-bit, 12-bit)

Also they can be cast into higher precision format and with the right compression it should not much matter, but it would still be a great addition

njaard commented 1 week ago

Hi @TKaluza , thank you for your feedback.

Can you describe your use case in more detail?

TKaluza commented 5 days ago

Hi @njaard,

Thank you for your response!

Certainly, let me elaborate. When working with various types of sensors, such as those used in microcontrollers (MCUs) or industrial environments, the data they produce can have varying bit depths. For instance:

1D sensors like temperature or pressure sensors often operate at lower bit depths (e.g., 16-bit integers). Line scanners or cameras, which might be closer to 2D data sources, may also use bit depths outside the standard range (e.g., 10-bit, 12-bit). Typically, we retrieve sensor data using protocols like Modbus, vendor-specific serial communication, OPC UA, or other similar techniques. Your library appears to be perfectly suited for scenarios where time-based sensor data (1D or possibly 2D) is continuously collected, processed on an edge device, and subsequently transmitted either incrementally or in chunks for further processing downstream.

Adding support for more diverse data types (such as 16-bit or even arbitrary bit depths) would help cover these use cases more comprehensively. While data can sometimes be cast to higher precision formats, doing so often comes with trade-offs in terms of storage efficiency and computational overhead, especially in environments with constrained resources.

Does this clarify the use case, or would you like me to provide more detailed examples? 😃