This package provides functions and classes to run wodan style processing functions in the Stactics AICore environment.
Use
pip install corebridgeto install corebrdige.
Wodan is a proprietary backend service that applies high performance, custom analytical processing to timeseries data in the Whysor data and dashboarding environment.
Each wodan module defines one function that operates as the entry point. The parameter annotations in this function definition are used to format data and retrieve parameters from the originating call to the wodan api. This function is called with data retrieved according to a specification and with additional parameters as annotated.
A simple function might look like:
import numpy as np
def multiply(data:np.ndarray, multiplier:float=1.0):
return data * multiplier
Wodan binds this function to a service endpoint and takes care of fetching data and parameters and converting the result for the caller.
For AICore users define a class, always named CustomModule with a
constructor __init__ and a method infer.
This package defines a baseclass to quickly construct a CustomModule
class that is able to use a wodan processor function inside the AICore
system:
import numpy as np
import corebridge
def multiply(data:np.ndarray, multiplier:float=1.0):
return data * multiplier
class CustomModule(corebridge.aicorebridge.AICoreModule):
def __init__(self, save_dir, assets_dir, *args, **kwargs):
super().__init__(multiply, save_dir, assets_dir, *args, **kwargs)
That’s it. Well, you can add parameters to __init__ that can be used
as hyperparameters in the web-interface and you could override infer
for the same reason. The baseclass takes care of converting call
parameters and data to the function specification and, calls the
function and converts the result for the caller, similar to the original
Wodan service.
This library is developed with NBDev - a literate programming toolkit that supports developing code using jupyter notebooks and mix code with documentation.
Literate programming is a methodology - introduced in 1984 by Donald Knuth - that combines a programming language with a documentation language. In this approach, a program is explained in a human language (such as English) alongside code snippets. The literate source file is then processed by a preprocessor to produce both source code and formatted documentation.
This paradigm enhances program robustness, portability, and maintainability, making it a valuable tool in scientific computing and data science[^1]
Documentation is prepared from the notebook with Quarto. Quarto too combines code with documentation but it does not extract source code into modules like nbdev.
Quarto uses Pandoc and, for pdf format, LaTeX. These must be available on your system.
Install Quarto as you see fit, there is a VSCode extension which handles this.
NBDev is available as PyPi package and is installed with
pip install nbdevor if you are using conda
conda install -c fastai -y nbdevIf so desired you can let NBDev install Quarto with
nbdev_install_quartoBut this ask for the system admin password.
Setup a virtual environment, activate it and install the development package and dependencies with, on linux
pip install -e ‘.[dev]’or on Windows
pip install -e .[dev]The above pip install should also install jupyter but to use it the kernel needs to be installed with:
python -m ipykernel install --user --name=corebridge.venvThe latter performs - nbdev_export - nbdev_test - nbdev_clean - nbdev_readme
Then commit and to upload to Pypi with nbdev_pypi