framework interface expansion

[laroque]

(Note, this issue is connected to nymph issue 7)

There has been some renewed discussion of adding the ability to interface with katydid's classes in an interactive way, ideally via python. I found the following demo of boost's python interface which may provide a good way to go about implementing this. Should investigate viability

This one looked promising but didn't work on the first try: https://www.preney.ca/paul/archives/107

This one did work: https://feralchicken.wordpress.com/2013/12/07/boost-python-hello-world-example-using-cmake/

Lessons learned:

• the first example hard-codes python 2.6 when adding include path details
• the second example shows removing the default "lib" prefix on the lib file, which makes the name actually be what we want to import in python
• the second example places the code for doing the python wrapping in its own source. We should discuss if structurally we want to do this or include it in the existing source for wrapped classes, presumably with some sort of pre-processor option that allows cmake to disable the python bindings as a feature?

Questions:

• is building python bindings an option of the katydid build?
• should it be some other repo with the wrappers? (I don't think so)
• should it be done first in nymph?
• what is the right bite-sized chunk to get a meaningful demo of this idea without having to do everything before it is ready?
• for build purposes, do we care what version of python we're using or does boost do whatever is there? if it matters, what version is our most desired to support (I suggest either 2.7 or 3.x where 2<=x<=4)

[laroque]

More complete examples including working in classes, can be found here: https://wiki.python.org/moin/boost.python

[laroque]

Some new notes (perhaps this thing should break into smaller issues):

• actively working on the feature/python branch both here and in nymph
• currently working specifically with python 2.7; would be nice to make this flexible, allowing a user to configure a version in cmake and building the python lib in a proper subdir of lib
• There is a scarab issue, would be nice to add -Wl,-z,defs to the linker flags for shared libraries (in all modes? in debug certainly; as an independent option from the build type?)

New questions I'm thinking about:

• Should the python interface use the katydid internals for building a sequence of processors and then executing them, or should one expect to write a for loop in python that calls the slot methods of each processor? Is there some middle ground?
• Do we want/expect users to develop complete new processors with custom data types in python, if they want to develop/test new components in python? This would allow the component to plug in anywhere in the analysis but seems hard (how do I make a new C++ type in my python?). Is there an easier first step if, say, we assume that python components will always come last and not pass anything back to C++?
• What classes do we actually want/need to wrap? In the case of the egg reader I've done KTEgg3Reader, KTRSAMatReaer, and KTEggProcessor; in hind sight it may only be this last one which needs a python interface, since the rest may not ever need to be used directly.
• What namespace/hierarchy of classes do we want to put in place within katydid to keep things reasonably organized and where do the organizational source/header files go?

[laroque]

Some very helpful insights here: http://stackoverflow.com/questions/30463563/boostpython-class-with-function-templates-how-to-add-instances-from-the-outsi

And here: http://stackoverflow.com/questions/5357411/export-template-function

[laroque]

yet another link to keep track of: https://skebanga.github.io/embedded-python/