*** Chipi - A (House) Automation for Common Lisp
!!! Still in alpha state, API design changes are probably required !!!
**** Motivation
I use home automation in my home. Currently utilizing [[https://www.openhab.org/][openHAB]]. While openHAB has many features and supports many devices and so on, it is also a heavy beast.
So my aim was to eventually replace openHAB. There are a few things still missing to do this. The current state of the Chipi project lacks the following for me to be able to replace openHAB:
**** What does Chipi currently do
***** Chipi consists of the following components:
item: items are a primitive to represent the temperatur of a sensor, the state of light in your house, the state of your window being open or closed, etc.
binding: bindings are a means to update an items value or distribute a value change to somewhere
persistence: persistences allow to persist and load the state of items. Currently two types of persistences are supported:
rules: rules are kind of scripts that are run on certain 'triggers'. Triggers may be a change of an item value, or a cron timing.
**** How does it work
Chipi is largely based on [[https://github.com/mdbergmann/cl-gserver][Sento]], a Common Lisp Actor Framework where features like thread-safety, queuing and an event-bus are very handy for Chipi.
How it works is to define and combine all the mentioned primitives in a file. This file is a Common Lisp source file. The definition of the primitives (item, binding, ...) is done using a defined DSL (domain specific language) which largely consists of a few macros. But it is also possible to define your own macros, functions or variables.
A full production example that defines items, bindings, persistences and rules can be seen [[https://github.com/mdbergmann/cl-etaconnector/blob/master/eta-hab.lisp][here]]. This script, very much based on Common Lisp, is a full example configuration of Chipi. At the end of the script evaluation of all items, persistences and rules will start their work.
This particular example uses Common Lisp libraries like [[https://github.com/bendudson/py4cl][py4cl]] (to talk to ina219 analog 2 digital converter for a pressure sensor in a cistern plugged to the GPIOs of a Raspberry PI 4), [[https://github.com/edicl/drakma][Drakma]] to query data of a [[https://www.shelly-support.eu/][Shelly]] power switch or [[https://github.com/snmsts/cserial-port][cserial]] to communicate with some other device to retrieve temperatur sensor data.
**** But let's go slowly
The top of the Chipi standalone script should include this (=eval-when=) in order to make sure that dependencies are loaded properly. But this is not mandatory if you find another way of loading dependencies.
(eval-when (:compile-toplevel :load-toplevel :execute) (ql:quickload :chipi)) ; at least this if you don't have more
Next thing is that you might want to define a separate package using =defpackage=, it allows to inherit symbols using =:use= or =:import-from= that you want to use in the script.
***** Defining the global environment The first thing Chipi specific that should be defined is the 'config'. This is done by =(defconfig)=. This expression should be before items, persistences and rules because it defines and starts the underlying actor system and sets up some other necessary structures.
***** Defining persistences Persistences are used to persist the value of an item and to retrieve it back. The influx persistence also allows to retrieve values in a certain time range.
** Simple persistence To define a 'simple' persistence you do:
(defpersistence :simple (lambda (id) (make-simple-persistence id :storage-root-path #P"some-path")))
The =:simple= here is the persistence id passed along to the lambda factory function.
The only information =simple-persistence= requires is where to store the item values. Each item is stored in its own file and just stores the last value. The value plus a timestamp (=GET-UNIVERSAL-TIME=) is stored in JSON format like this:
{"value":66.19999694824219,"timestamp":3907315980}
Under normal circumstances you don't need to deal with simple persistences directly. To retrieve an item value you'd call =ITEM:GET-VALUE=. Though it is possible to call =PERSP:FETCH= and retrieve the current value from the persistence directly.
** Influx persistence A persistence that allows retrieving item values of a time range, i.e. to build daily or weekly average values, is influx. Currently only influx v2 is supported. Define such persistence like this:
(defpersistence :influx (lambda (id) (make-influx-persistence id :base-url "http://your-host:8086" :token "your-token" :org "your-org" :bucket "your-bucket")))
To retrieve a range of values you can call =PERSP:FETCH= with an optional range instance. Currently only =RELATIVE-RANGE= exists.
**** Bindings Bindings are a means to interact with sources or targets, meaning they allow interactivity with the item value. Bindings are defined as part of an item definition and not* on toplevel. A basic binding definition looke like this:
(binding :initial-delay 5 :delay 60 :pull (lambda () 0) ;;pull value from somewhere :transform (lambda (value) (1+ value)) :push (lambda (value)) ;; push to somewhere else :call-push-p t)
This binding uses the =pull= function to retrieve a value, which is passed on to the item value. When to =pull= is determined by =:initial-delay= and =:delay= in seconds where the former is an 'initial delay' and the latter a repetetive delay. =:call-push-p= actually defines whether the =push= function is called when the value was updated. The =push= function can be used to push the value elsewhere if required. Both =pull= and =push= are optional. Though one of the two should be used, otherwise the binding doesn't make much sense. What is =transform=? It is optional but can be used to transform the value retrieved with =pull=. =transform= should return a transformed value.
Thinking further, I'd like to have bindings that are specific to pulling from http, serial, or whatever, and allow to be specified in that way. The =pull=, =push= functions are very generic but may require repetition and are not enough specialized.
See next how to define and attach bindings on items.
***** Defining items The simplest form to define an item is:
(defitem 'myitem "My Item" 'integer)
This defines a plain item that can hold a value. You could manually use =SET-VALUE= function to give it a value or =GET-VALUE= to retrieve its value. In some cases this is useful in 'rules'. See later.
The three parameters define an id of the item (for easier lookup), a label and a type hint. The type hint is not necessary (can be specified as =NIL=) unless you want to use influx db where under the hoods it is necessary to bring the value in the right format based on what type the value is in. Checkout [["https://github.com/mdbergmann/chipi/blob/main/src/persistence-influx.lisp"][influx persistence]] for which types are supported. However, even if not required it might be a good idea to define the type for clarity.
Usually you'd want to at least define an initial value. You can do so by:
(defitem 'myitem "My Item" 'integer :initial-value 0)
** Define and attach bindings
In many cases you want to retrieve the item value from somewhere and maybe also want to push it somewhere else once it was set. This is where bindings come in. There can be more =binding= definitions on an item but this only really makes sense if you plan to =push= to more places. An item definition with binding looks like this:
(defitem 'myitem "My Item" 'integer :initial-value 0 (binding :initial-delay 0.1 :delay 30 :pull (lambda () (do-some-http-get)) :push (lambda (value) (do-some-http-post)) :call-push-p t))
** Attaching persistences on the item definitions
Persistences, as defined above can now be 'attached' to the item like this:
(defitem 'myitem "My Item" 'integer :initial-value 0 (binding :initial-delay 0.1 :delay 30 :pull (lambda () (do-some-http-get)) :push (lambda (value) (do-some-http-post)) :call-push-p t) :persistence '(:id :simple :frequency :every-change :load-on-start t) :persistence '(:id :influx :frequency :every-20s))
It is possible to attach multiple. In the case above both have different purposes. The =:simple= persistence is used to just store the latest value and can recover from it when told so using =:load-on-start=.
The =:influx= persistence will just store every value change to the database.
The =:frequency= defines how often the value is stored. =:every-change= will store the value to the persistence on every change of the item value. =:every-20s= (form example) stored the value every 20 seconds recirring. The notation here is =:every-N<s|m|h>= where N is the number, s (seconds), m (minutes) and h (hours).
***** Defining rules
Rules are scripts that are run on certain triggers. Triggers are the change of one or more item values or one or more cron definitions. Example:
(defrule "My Rule" :when-item-change 'my-item :when-cron '(:minute 0 :hour 0) :do (lambda (trigger) (case (car trigger) (:item (let ((item (cdr trigger))) (format t "Item changed: ~a~%" item) ;; asynchronously do something with the value (future:fcompleted (item:get-value item) (value) (do-some-action-with-value value)))) (:cron (format t "Cron triggered: ~a~%" (cdr trigger)) (do-some-action))))
Rules can be triggered by item value changes. To subscribe to certain items one has to use =:when-item-change= with the item id of the item definition. Use multiple =:when-item-change= to subscribe to multiple item changes.
The other trigger is cron. The lowest granularity is minutes. Specify cron triggers with =:with-cron=. Also multiple triggers can be defined. The =cdr= of the trigger variable is the cron expression.
When a cron trigger is specified as '=(:boot-only t)= then this means that the rule is called immediately after initialization, but only once.
**** Redefining persistences, items and rules
Those elements can be redefined, meaning re-evaluated to update a changed configuration. The re-evaluation usually happens in Emacs in the same way as functions are re-evaluated using =C-c C-c= or by reloading the whole script.
Caveeat: if a persistence is changed and re-evaluated also the items where it is attached have to be re-evaluated.
**** Logging
The project uses log4cl. You can change the log level for '=(chipi)= to suite your needs of granularity. The underlying Sento actor framework will log extensively on =:debug= level so a good idea is to silence this via =(log:config '(sento) :warn)=.