In-memory Python interface

CADET already provides a MATLAB interface via MEX-functions. A similar in-memory interface can be done in Python. Such an interface reduces the overhead of reading input files, allocating memory, configuring and initializing the simulation. This is especially interesting when doing optimization or repeated simulations (e.g., parameter studies).

There are basically two approaches to build an in-memory interface:

Intrusive: Expose C++ API via boost::python, Cython, CPython and directly interact with Python objects
Nonintrusive: Expose a generic C API that is used from the Python side via ctypes and use callback functions when necessary

Generally, the file format should be recreated on the Python side using nested dictionaries.

Attention: Due to the handling of AD directions, whose active number is reset when starting time integration, running simulations concurrently in the same (Python) process / libcadet instance might produce wrong sensitivities or results (when using AD for Jacobians) under specific circumstances. This is the case when the simulations use a different number of active AD directions and the one with less directions is started last. The global active-AD-directions-variable is then reset to the lower number, which causes the other simulation to use the same number of directions.

Intrusive approach

Write a ParameterProvider that reads nested Python dict structures to get the parameters into CADET.
Implement a writer class that turns arrays into numpy arrays (replacing HDF5Writer, for example).
Expose a reasonable set of commands in a Python object (maybe the same methods as in Matlab MEX).

Pros / cons:

Specific to Python, requires additional dependencies
Maybe faster to implement

Nonintrusive approach

Plain C wrapper for C++ objects or at least most used functions (cf. MATLAB interface)
Pass model setup as hardcoded nested C structs or use a callback-based ParameterProvider
Memory is managed by CADET and will need to be released explicitly (by calling C functions)
Python object that handles everyting and takes care of communication (maybe called Simulator)

Pros / cons:

Rather general, might be used for interfacing with other languages (e.g., Julia)
Does not require additional dependencies
C structs are one more point in code to update when changing stuff
Callback-based ParameterProvider is probably slow (Python -> C -> C++ -> Python roundtrip)
Separation between CADET library and Python client via callback "protocol".
Can be directly built into libcadet without introducing another binary / CMake target.

Proposal: Nonintrusive approach using callbacks

A minimal working setup:

Implement callback-based ParameterProvider: Takes ctype function pointers to callbacks that return values or arrays (including their size).
Create an exported C function that simply creates a Driver, configures it and runs the simulation.
Create an exported C function that cleans up (delete Driver, release memory).
Specify a model in Python via nested dicts containing mostly numpy objects (even for scalars?).
Plot the results by wrapping a numpy.array around the returned array pointer.

Details on the callback-based ParameterProvider:

Two possible query styles: readDouble(string fullPathToItem) or pushGroup(string groupName), readDouble(string itemInCurrentGroup), popGroup().
Depending on the query style, tracking of the current group / scope is handled in C++ or in Python, respectively.
It is easily possible to have both styles implemented and switchable at runtime.
Function pointers need to be passed from the Python side upon construction of the ParameterProvider (or maybe later). Those function pointers could be collected in a C struct since this is fixed and unlikely to change.
Signature of the callbacks could be bool readXYZ(string itemOrPath, xyz* out) for scalars and bool readXYZ(string itemOrPath, xyz* out, int* size) for arrays. The bool return value signals whether the key has been accessed successfully.

Estimated timeframe for a working prototype: 1.5 months.

modsim / CADET