llr-cta / calin

IACT calibration framework
GNU General Public License v2.0
4 stars 3 forks source link

SWIG interface vs. google protoc ? #127

Closed dneise closed 6 years ago

dneise commented 6 years ago

I have a question regarding the SWIG generated python interface you describe here very nicely

https://github.com/llr-cta/calin/wiki/Python-Protobuf-interface

and the Python interface generated by google using protoc as e.g. explained in the protobuf Python tutorial like this?

protoc --python_out=telescope_event.py telescope_event.proto

What is the difference between the two Python interfaces? Is one simpler to understand than the other? Or is one faster?

sfegan commented 6 years ago

Hi Dominik,

My code almost exclusively uses C++ classes to do computation, mainly for speed reasons, but I find it tedious to write driver code in C++, preferring to run the code from python scripts & notebooks... I think this is common these days, and probably you do the same on FACT. I build the Python-C++ interface in SWIG, which is much less tedious than doing it by hand.

My C++ code uses protoc for many of the data structures - they accept pointers/references to C++ protoc structures as input and return them as output. Therefore I need to have the protoc structures available to the SWIG interfaces for the C++ computation classes - in other words they have to be "within" the SWIG universe for my package, i.e. they have to be compiled by swig.

The direct Python interfaces generated by "protoc --python_out" (either the pure python versions or the newer C++ interface ones) produce classes that work fine in Python, but they are classes that SWIG has no idea about.. and so they can't be passed to the C++ computation classes through the SWIG interfaces.

The main reason that I wrote the SWIG extension to protoc was therefore to bring the protoc classes into my SWIG universe. I initially tried to have SWIG parse the C++ header files generated by protoc but that didn't work at all as the header files are too complex for SWIG to understand. My SWIG protoc extension generates very simple interfaces to the protoc classes that SWIG can understand. I expect that they are basically just as fast as the the google "C++ interfaces" but I never really tested them, as in my code the computation is really done in the C++ classes themselves.. the Python is just used for plotting / control etc.. and it doesn't really matter how fast the interface is.

Hope that was some help, Steve