c685ea27-59ea-48a1-946a-0df4e95439aa
commented
15 years ago feature: extension module C++ howto/example in extending-embedding/c-api documentation
why: The embedding/extension documentation states that module implementation in c++ is possible, without providing any guidance beyond this. Coders more familiar/comfortable with c++ than c, writing c++ code to expose in python, might want to create the actual python class/module implementations in c++ classes or structs. A basic guide can help prevent wastage of energy reinventing the wheel, and also serve as a guide to safe/preferred style. The method outlined here can be a starting point in finding out what that style is. Also it seems, to my eyes at least, a little easier to 'visualise' and apply than the equivalent in plain C (the Noddy module etc), a bit more 'systematic', and easier to understand the relation between c and python objects. So after some trial and error, discovering the pitfalls, I am finding it easy to create useful python classes with this recipe. Others might also find it useful, even if only as a stimulus to find a Better Way.
Python is an object-oriented language noted for its clarity, so an allegedly simple and clear OO implementation strategy for module classes should be examined.
what: I've written a module currently called 'cpeepee', containing a basic python class ,(struct TestRealDestructor in C++, 'cpeepee.destructo' in python). It has been tested with python-2.5 and g++-4.2 on ubuntu-'hardy heron' and python-2.5/g++-3.4 on FreeBSD-6.4.
The c++ struct inherits from PyObject.(You could also inherit from PyVarObject or other more specialised types). The most important non-feature of this class is that it is not virtual, has no virtual destructor or functions(in member objects virtual destructors and the rest are OK however). Therefore the packing of ob_refcnt and friends is correct, and casting to/from PyObject is safe(-and otherwise is not - the PyObject cast from such a type with virtual destructor is offset by the size of a pointer -the vptr?-, but when python casts back to the type - from void or so?- the offset is not removed, leading to mayhem).
[You could also not inherit, simply put the PyObject_HEAD macro as the
first entry in the struct - could be a class also, as long as the
PyObject members(ob_refcnt, ob_type ... )were public - You would have to
write a new macro to fill in those members in the constructor's
initialiser list, but that doesn't look too hard. As it is the inherited
classes use a function and some macros (almost identical to
PyObject_HEAD_INIT(typo) ) to fill in the PyObject parent. Again, vfuncs
are out]The destructo method and member tables are static members ofTestRealDestructor, as is its type object.(Other optional tables etc should also be static members if provided - makes for a simple consistent setup) The objects constructor is called from TestRealDestructor::type.tp_new() and passed the args and kwds arguments it is passed, using placement new with memory obtained from tp_alloc()(see TestRealDestructor::create() ). Being able to properly call an object's constructor was the real motivation for writing the code this derives from. Using C style one is stuck casting a char* to your type and then filling its fields by hook or crook.
On error, either you could throw a c++ exception in constructor to catch in tp_new, and convert to a python exception, or simply throw the python exception in the constructor, and then check if PyErr_Occurred() in tp_new() - the approach with destructo.
Since tp_new and the constructor take care of object creation, this leaves tp_init not doing much at all. It could be used for extra checks, or printing funky messages about preserved ham
Functions to be exposed in python API as class member functions should static members of the class, and to do the work they call ordinary member functions of the class object passed into the static function . You could use global static functions(with the first arg TestRealDestructor say, rather than PyObject), but that's less clear, less systematic, less OO.
Everything can be placed in a namespace to avoid any pollution of global namespace.
I've worked out the bugs that were obvious to me, so it should compile and run as is, without error messages. The one compile warning I don't know how to banish comes from the offsetof macro when setting member offsets in the member table(as copied from the c example) - however the object whose offset is so established works fine from python, so I think it's a spurious warning.
If you find any issues with my approach, I'm happy to work through them, or if you know what to do then make whatever changes you think required.
I'm happy to put any code or words I contribute on this topic to go under python's copyright as long as I'm credited somehow(however you normally do that). Whatever, I'm happy to contribute to such a great project as python
sincerely
John O'Driscoll
amauryfa
terryjreedy
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields: ```python assignee = None closed_at =
created_at =
labels = ['docs', 'performance']
title = 'c++ extension module implementation guide/example in extending/embedding documentation'
updated_at =
user = 'https://bugs.python.org/subgeometer'
```
bugs.python.org fields:
```python
activity =
actor = 'terry.reedy'
assignee = 'docs@python'
closed = True
closed_date =
closer = 'terry.reedy'
components = ['Documentation']
creation =
creator = 'subgeometer'
dependencies = []
files = ['14253']
hgrepos = []
issue_num = 6251
keywords = []
message_count = 4.0
messages = ['89190', '89193', '89237', '112914']
nosy_count = 5.0
nosy_names = ['georg.brandl', 'terry.reedy', 'amaury.forgeotdarc', 'subgeometer', 'docs@python']
pr_nums = []
priority = 'normal'
resolution = 'later'
stage = None
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue6251'
versions = ['Python 2.7']
```