search

zephyrproject-rtos / zscilib

An open-source scientific computing library for embedded systems running Zephyr OS or standalone.
https://zephyrproject-rtos.github.io/zscilib/
Apache License 2.0
130 stars 24 forks source link
arm embedded module zephyr

readme

zscilib

Zephyr Scientific Library (zscilib)

API Documentation

The Zephyr Scientific Library (zscilib) is an attempt to provide a set of functions useful for scientific computing, data analysis and data manipulation in the context of resource constrained embedded hardware devices.

It is written entirely in C, and while the main development target for the library is the Zephyr Project, it tries to be as portable as possible, and a standalone reference project is included to use this library in non-Zephyr-based projects.

This version of zscilib has been developed and tested against Zephyr 3.3.0.

Motivation

As the processing power of small, embedded MCUs increases and costs fall, more computation can be done on the endnode itself. This allows for more of the 'complex' data analysis that used to take place at the PC or server level (data aggregation, statistical analysis, etc.) to be done in less time, using less data storage, and at a lower overall processing cost on small, embedded devices.

A key goal of zscilib is to allow more data processing to happen on the endnode.

By generating scientifically-relevant data points (standard SI units, pre-filtered data, etc.) directly on the endnode, zscilib aims to be a bridge between raw data and more numerically complex toolkits like gsl, numpy or R.

What makes zscilib distinct?

Numerous high quality, mature, open source scientific libraries already exist:

Despite the wealth of mature functions in these existing libraries, though, they tend to have the following two problems in an embedded context:

The second item is of particular importance, since the goal of embedded systems is often 'sensing' via raw data, correlating that data, and acting on the final data points or passing them on for further analysis.

CMSIS-DSP contains a number of highly efficient algorithms for filtering raw sensor data, but it doesn't offer any domain-specific assistance converting filtered accelerometer vectors into orientation data, for example, or reading a set of photodiodes and converting that data into a useful photometric value like lux. It is excellent at 'conditioning' data, but not at 'understanding' it.

zscilib aims to find a middle ground between these two, allowing for richer processing of raw data, but within the confines and limitations of the class of microcontrollers commonly used on low-cost sensor endnodes.

Quick Start: Standalone

A few makefile-based projects are included in samples/standalone showing how zscilib can be used independent of Zephyr.

If you already have an appropriate GNU toolchain and build tools (make, etc.) installed, you can simply execute the following commands:

$ cd samples/standalone/svd_pinv
$ make
$ bin/zscilib
  Hello, zscilib!
  ...

Quick Start: Zephyr RTOS

Running a sample application

To run one of the sample applications using qemu, run the following commands:

Be sure to run source zephyr/zephyr-env.sh (OS X or Linux) or .\zephyr\zephyr-env.cmd (Windows) before the commands below! This also assumes qemu-system-arm is available on your local system.

$ west build -p -b mps2/an521/cpu0 \
  modules/lib/zscilib/samples/matrix/mult -t run
...
*** Booting Zephyr OS build zephyr-v2.6.0-536-g89212a7fbf5f  ***
zscilib matrix mult demo

mtx multiply output (4x3 * 3x4 = 4x4):

14.000000 17.000000 20.000000 23.000000 
35.000000 44.000000 53.000000 62.000000 
56.000000 71.000000 86.000000 101.000000 
7.000000 9.000000 11.000000 13.000000

Press CTRL+A then x to quit qemu.

Running Unit Tests

To run the unit tests for this library, run the following command:

$ twister --inline-logs -p mps2/an521/cpu0 -T modules/lib/zscilib/tests

See the tests folder for further details.

To run compliance tests to make sure submitted code matches Zephyr PR requirements, run this (updating HEAD~2 for the number of commits to check, or setting it to origin/master.. to check everything):

$ ../../../zephyr/scripts/ci/check_compliance.py \
  -m Gitlint -m Identity -m Nits -m pylint -m checkpatch \
  -c HEAD~2..

Debugging with QEMU

If you wish to debug using QEMU (and with minor variation actual hardware), you can run the following commands to start a new GDB debug session.

For an overview of debugging in GDB, you may find the following presentation useful: LVC21-308: Essential ARM Cortex-M Debugging with GDB

In one terminal window, run:

$ west build -p auto -b mps2/an521/cpu0 modules/lib/zscilib/samples/matrix/pinv

Once the ELF file has been built, we can start a GDB server on the default 1234 socket, and wait for a new connection via:

$ cd build
$ ninja debugserver

In a new terminal window, connect to the GDB server via:

$ cd <zephyr_path>
$ arm-none-eabi-gdb-py \
  --eval-command="target remote localhost:1234" \
  --se=build/zephyr/zephyr.elf

The -py extension is optional, and makes use of a version of GDB from the ARM GNU toolchain releases that enables Python scripts to be used with your debug sessions. See the LVC21-308 presentation at the top of this section for details.

From here, you can start debugging with the (gdb) prompt.

For example:

(gdb) b main
(gdb) c
Continuing.

Breakpoint 1, main () at modules/lib/zscilib/samples/matrix/pinv/src/main.c:70
70              printf("\n\nzscilib pseudo-inverse demo\n\n");
(gdb) n
72              pinv_demo();
(gdb) step
pinv_demo () at modules/lib/zscilib/samples/matrix/pinv/src/main.c:25
25              zsl_real_t vi[18 * 3] = {
(gdb) n
... 
(gdb) quit

Floating-Point Usage

zscilib can be configured to make use of single-precision (32-bit) or double-precision (64-bit) floating point values via the CONFIG_ZSL_SINGLE_PRECISION flag, which will determine the size of zsl_real_t used throughout the library. The default setting for this flag is n, meaning 64-bit values are used by default.

There is a tradeoff between the added range and precision that 64-bit (double-precision) floating point values offer, and the memory and performance gains of the smaller, less-precise but faster 32-bit (single-precision) operations.

Due to the reduced precision of single-precision values, certain complex functions in zscilib are only available when double-precision is enabled (PINV, SVD, etc.).

Comparison

Single-Precision (32-bit) Floats

Double-Precision (64-bit) Floats

Float Stack Usage in Zephyr

The sample code in this library typically has the CONFIG_FPU option set, meaning that floating-point support is configured for Unshared FP registers mode. This mode is used when the application has a single thread that uses floating point registers.

If your application makes use of multiple threads, and more than one of these threads uses floating-point operations, you should also enable the CONFIG_FPU_SHARING config flag, which configures the kernel for Shared FP registers mode. In this mode, the floating point registers are saved and restored during each context switch, even when the associated threads are not using them. This feature comes at the expense of an extra 72 bytes of stack memory per stack frame (s0..s15 + FPCSR, plus an alignment word to ensure that the stack pointer is double-word aligned).

Current Features

Features marked with the v0.2.0 flag are in progress or planned as part of the current release cycle, and may be partially implemented or stubbed at present. v0.3.0 indicates features planned for that later release.

Linear Algebra

Vector Operations

Feature Func f32 f64 Arm Notes
Array to vector zsl_vec_from_arr x x
Copy zsl_vec_copy x x
Get subset zsl_vec_get_subset x x
Add zsl_vec_add x x
Subtract zsl_vec_sub x x
Negate zsl_vec_neg x x
Sum zsl_vec_sum x x 2 or more vects
Scalar add zsl_vec_scalar_add x x
Scalar multiply zsl_vec_scalar_mult x x
Scalar divide zsl_vec_scalar_div x x
Distance zsl_vec_dist x x Between 2 vects
Dot product zsl_vec_dot x x
Norm/abs value zsl_vec_norm x x
Project zsl_vec_project x x
To unit vector zsl_vec_to_unit x x
Cross product zsl_vec_cross x x
Sum of squares zsl_vec_sum_of_sqrs x x
Comp-wise mean zsl_vec_mean x x
Arithmetic mean zsl_vec_ar_mean x x
Reverse zsl_vec_rev x x
Zero to end zsl_vec_zte x x 0 vals to end
Equality check zsl_vec_is_equal x x
Non-neg check zsl_vec_is_nonneg x x All values >= 0
Contains zsl_vec_contains x x
Quicksort zsl_vec_sort x x
Print zsl_vec_print x x

Matrix Operations

Feature Func f32 f64 Arm Notes
Array to matrix zsl_mtx_from_arr x x
Copy zsl_mtx_copy x x
Get value zsl_mtx_get x x
Set value zsl_mtx_set x x
Get row zsl_mtx_get_row x x
Set row zsl_mtx_set_row x x
Get col zsl_mtx_get_col x x
Set col zsl_mtx_set_col x x
Add zsl_mtx_add x x
Add (d) zsl_mtx_add_d x x Destructive
Sum rows zsl_mtx_sum_rows_d x x Destructive
Sum rows scaled zsl_mtx_sum_rows_scaled_d x x Destructive
Subtract zsl_mtx_sub x x
Subtract (d) zsl_mtx_sub_d x x Destructive
Multiply zsl_mtx_mult x x
Multiply (d) zsl_mtx_mult_d x x Destructive
Multiply sc (d) zsl_mtx_scalar_mult_d x x Destructive
Multiple row sc (d) zsl_mtx_scalar_mult_row_d x x Destructive
Transpose zsl_mtx_trans x x
Adjoint 3x3 zsl_mtx_adjoint_3x3 x x
Adjoint zsl_mtx_adjoint x x
Wedge product zsl_mtx_vec_wedge x
Reduce zsl_mtx_reduce x x Row+col removal
Reduce (iter) zsl_mtx_reduce_iter x x Iterative ver.
Augment zsl_mtx_augm_diag x x Adds row+col(s)
Determinant 3x3 zsl_mtx_deter_3x3 x x
Determinant zsl_mtx_deter x x
Gaussian El. zsl_mtx_gauss_elim x x
Gaussian El. (d) zsl_mtx_gauss_elim_d x x Destructive
Gaussian Rd. zsl_mtx_gauss_reduc x x
Column norm. zsl_mtx_cols_norm x x Unitary col vals
Gram-Schimdt zsl_mtx_gram_schmidt x x
Elem. norm. zsl_mtx_norm_elem x x Norm vals to i,j
Elem. norm. (d) zsl_mtx_norm_elem_d x x Destructive
Invert 3x3 zsl_mtx_inv_3x3 x x
Invert zsl_mtx_inv x x
Balance zsl_mtx_balance x x
Householder Ref. zsl_mtx_householder x x
QR decomposition zsl_mtx_qrd x x
QR decomp. iter. zsl_mtx_qrd_iter x
Eigenvalues zsl_mtx_eigenvalues x
Eigenvectors zsl_mtx_eigenvectors x
SVD zsl_mtx_svd x
Pseudoinverse zsl_mtx_pinv x
Min value zsl_mtx_min x x
Max value zsl_mtx_max x x
Min index zsl_mtx_min_idx x x
Max index zsl_mtx_max_idx x x
Equality check zsl_mtx_is_equal x x
Non-neg check zsl_mtx_is_notneg x x All values >= 0
Symmetr. check zsl_mtx_is_sym x x
Print zsl_mtx_print x x
Unary matrix operations

The following component-wise unary operations can be executed on a matrix using the zsl_mtx_unary_op function:

Binary matrix operations

The following component-wise binary operations can be executed on a pair of symmetric matrices using the zsl_mtx_binary_op function:

NOTE: Component-wise unary and binary matrix operations can also make use of user-defined functions at the application level if the existing operand list is not sufficient. See zsl_mtx_unary_func and zsl_mtx_binary_func for details.

Numerical Analysis

Statistics

[1] Only available in double-precision

Probability Operations

Interpolation

Physics

Atomic

Dynamics

Electrical Components

Electric

Electricity

Energy

Fluids

Gases

Gravitation

Kinematics

Magnetics

Mass

Momentum

Optics

Photons

Projectiles

Relativity (v0.3.0)

Rotation

Sound

Thermodynamics

Waves

Work

Motion and Orientation

AHRS/Attitude (Degrees)

Compass

Euler Angles (Radians)

Gravity

Quaternions

Sensor Fusion

Colorimetry

Types/Structs

Functions

Color Data

Illuminants
CIE Standard Observer Models
CIE Luminous Efficiency Functions
XYZ to RGB Color Space Correlation Matrices

Chemistry

Measurement API (v0.2.0)

The zsl_measurement struct is a proof of concept attempt at representing measurements in a concise but unambiguous manner.

It consists of:

There is an option to adjust the measurement's scale in +/- 10^n steps (Scale Factor) from the default SI unit and scale indicated by the SI Unit Type. For example, if 'Ampere' is indicated as the SI unit, the measurement could indicate that the value is in uA by setting the scale factor to -6.

Longer Term Planned Features

Help is welcome on the following planned or desirable features.

Scalar Operations

Digital Signal Processing (v0.3.0)

Spectrometry

Calibration

Architecture-Specific Optimisations

Basic tooling has been added to allow for optimised architecture-specific implementations of key functions in assembly.

At present, this feature isn't being actively used or developed, but an aim of zscilib is to add optimised versions of key functions to try to get the best possible performance out of limited resources.

Initial optimisation will target the Arm Cortex-M family of devices and the Thumb and Thumb-2 instruction sets, though other architectures can be accommodated if necessary or useful.

Code Style

Since the primary target of this codebase is running as a module in Zephyr OS, it follows the same coding style, which is itself based on the Linux kernel coding style.

You can format the source code to match this style automatically using the uncrustify command line tool, which has plugins available for many common text editors (Atom Beautify, for example).

Contributing

If you wish to contribute to this library, you can raise a PR as follows:

  1. Fork the repository: https://github.com/zephyrproject-rtos/zscilib/fork
  2. git clone your forked repository.
  3. Update your local repo and commit any changes.
  4. Push the changes out to your fork on Github.
  5. Navigate to https://github.com/zephyrproject-rtos/zscilib and to the right of the Branch menu click New pull request.
  6. Fill out the form that is presented.
  7. Click the Create Pull Request button to submit the PR.

Also have a look at the Issues page to see if there is any outstanding work or issues that you might be able to help with!

License

Apache 2.0.