Feat: callbacks

[liyier90]

Enables users to attach callback functions to nodes to be run at certain points/events in the pipeline.

Implemented events:

• run_begin: Before Node.run()
• run_end: After Node.run()

Supported callback types:

• Pure function
• Class methods
• Static methods
• Instance methods (require instantiating the object in the callback script)

UI: Users will specify callbacks for the node under the callbacks config key. Each callback is specified as:

callback definition = module part, "::", callback part
module part         = { subdir name, "." }, script name
callback part       = function
                     | class name, "::", method
                     | instance name, "::", instance method

script name     = ? name of callback Python script ?
subdir name     = ? name of individual subdirectory folders relative to `callbacks` folder ?
class name      = ? name of class which contains the callback methods ?
instance name   = ? name of instantiated object which contains the callback method ?
function        = ? name of callback function ?
method          = ? name of class/static methods ?
instance method = ? name of instance method ?

Incorrect callback definitions are skipped with a warning log, program is not halted.

Implementation detail: In _edit_config() of both AbstractNode and DeclarativeLoader, avoid going deeper into nested dictionary if the key is callbacks. This allows us to define the callbacks config in PeekingDuck as

callbacks: {}

and in pipeline config as

- node.name:
    callbacks:
      run_begin: ["callback::definition"]

This allows us to implement more callback events in the future without have to add the event key to all the config files in PeekingDuck.

Sample usage: Extract callbacks.zip into your project directory. Ensure that the callbacks directory is at the same level as dev_callback.yml. You should have the following directory layout:

project_dir
+-callbacks
| +-cb_dir
| +-my_callbacks.py
| \-sql_callback.py
+-dev_callback.yml
\-...

Run dev_callback.yml and you should see logging outputs such as

• called 1 from callback_1() in my_callbacks.py
• called z from cb_1() in cb_dir/mycb.py
• Suppressing logging from MyCallback.callback_2() in my_callbacks.py
• Exceed zone limit ... from MyCallback.callback_3() in my_callbacks.py
• Update db with frame_count ... from cb_obj.track_count_callback() in sql_callback.py

Additional changes:

• Refactored depracation code into separate methods (for ease of removal in the future)
• Removed key argument from obj_det_class_name_to_id()

[ongtw]

Can you write up a tutorial Using Callbacks under the Peaking Duck tutorial section on using callbacks and giving some sample codes/pipelines as illustrations?

[liyier90]

Added a "Using Callbacks" section to Peaking Duck tutorial. Includes:

• tutorial to go over the implemented events and callback definition
• tutorial to replicate the "Interfacing with SQL" tutorial, replacing the output.sqlite custom node with a callback