Generalization of backend API

[mrh1997]

While reasoning about my RPC library's design which shall be based on the parser of pyclibrary I came to the conclusion that your (really nice) concept of pluggable backends would be a perfect match for my library.

But according to my (yet) limited knownledge of the backend it has a drawback: it i too specialized for the current purpose. In fact it does two things:

• (dynamicially) generate python ctype objects from a C header file
• provide (really useful) extensions to ctypes (primarly the CallResult class)

If these features would be modularized and the python ctype object generation would be generalized to a static interface file generator, the range of applications of the library would be drasticially extended:

• It would be possible to write a backend for any language and for any ffi-library!!!
• The extensions (CallResult) could be swapped out into a separate module. This means you could use it even if you do not want to use CParser/CLibrary (i.e. if you do not have a C header file, but want to benefit from this Module either)
• projects which need to access a C library could do this easily without the need to add pyclibrary as dependency (pyclibrary would transform the header file once to a python module, which is added the project)

My idea of the static interface generator is basicially a template engine which gets the AST from the CParser and outputs a interface definition for a specific language/ffi-library/parametrization. If one still needs dynamic interface object generation (i.e. if the according C header file is not static; a use case which should barely occur), one could still generated the interface definition module as in-memory string object run exec() on this string.

To come to an example: The current backend would be replaced by a template, that converts the following header file:

struct sample_struct_t {
   int * a;
} ;
void sample_func(struct sample_struct_t * x);

to the follong python module:

import ctypes

lib = ctypes.CDLL('...')

class sample_struct_t(ctypes.Structure):
    _fields_ = [("a", ctypes.POINTER(ctypes.c_int))]

lib.sample_func.restype = ctypes.c_void
lib.sample_func.argtypes = [POINTER(sample_struct_t)]

(This example generated a python module without using CallResult. But of course it would also be possible to integrate CallResult into the generated module; would depend on the parametrization of the code generator)

@MatthieuDartiailh: What do you think about it? Regarding all the work to be done: as I need to do it anyway (for my RPC library, which has to compile a C-wrapper from C Headerf files) I would be glad to do it as part of pyclibary instead of my own project.

[MatthieuDartiailh]

Those ideas are very interesting I think. And I would be very happy to see your work become part of pyclibrary.

Currently a backend is meant to back a CLibrary by exposing the specificity of the chosen ffi. It is specific to C but not to ctypes. It makes user life easier by doing several things on top of the ffi :

• it make sure that a single binder per libary is in activity at a time (see metaclass)
• it generate the appropriate classes for any custom types defined in the header.
• it abstracts the way to create pointers and arrays.
• it wrap the functions to CFunctions which add unspecified pointers and always return CallResult (I really like the on the fly creation of pointers because it makes the library much simpler to use).
• it gives access to all the macros and enumeration (which are a pain to copy when creating a ctypes wrapper).

I think it would make sense to move to a kind of binding generations like the one you suggest. We could actually propose :

• pure ctypes bindings
• smart ctypes bindings (using clibrary and potentially packing it in the module).
• others such cffi bindings, cython or even C bindings (is that what you have in mind for your project ?) in both pure and smart versions (THIS WOULD BE AWESOME)

Creating an in memory wrapper (or calling a cached one) should remain easy (which would be equivalent to what we are doing now). In some cases you cannot version control the wrapper because it would expose proprietary headers .... (I have cases like that)

For the roadmap, I would say we first focus on the parser and the preprocessor (you mentioned somewhere extracting it from the parser which would make sense). Once this is done we can revisit the generation of the bindings.

[mrh1997]

I am happy that you like the concept. Of course I will finish all jobs discussed until now before starting the generalized backend mechanism. I create this issue that early, since we might need some time to discuss everything. When I am finished with my current work, I will know already how to continue then...

---

regarding your proposals of the new binding generator:

1. I assume that you meant with "smart bindings" bindings that includes all the "things on top of ffi" like CFunction and so on. In this case I propose a solution were we start with option 2 (smart ctypes bindings), since this is exactly what we have now => the new library would be compatible with 0.1.1.
2. It would be very less effort to make this binding generator parametrizable later, so that you can choose between pure and smart. This could be interesting for a lot of people who are used to work with ctypes (or have existing python modules based on ctypes) and dont want to switch to a new syntax. Especially for presenting the power of pyclibrary this could be nice, since everybody knows C syntax and ctypes. So people could quickly convince themselves of its power by checking their concrete corner cases.
3. The last idea would be doable, too. I would stick to plain old C extension modules, since this does not enforce another dependency. It is more effort, as the C source code of a C extension is a little bit complexer. The big drawback is, that it does not work on other python implementations like PyPy/IronPython/... And although the idea sounds awesome, I see no big advantage for the user apart from better performance (ctypes has some additional overhead compared to C extension modules).
4. Not mentioned by you but in my eyes the real gain: bindings for other languages with ffi support. Everybody not using C/C++/Objective C/D suffers from the same problem. With every additional language binding a whole language community will be added to the potential users (same effect as on IPython Notebook when it was opened to other languages and renamed to Jupyter). But this job has to be done by s.b. else ;-)
5. My personal interest is another use case: my binding generator will generate an c module to be compiled into my microcontrollers firmware. This c module will receive messages from a (pluggable) protocol and dispatch them to the firmware's internal c function. Furthermore a python module will be generated, that does the same job vice versa (python functions, that generate messages which are send to the microcontroller then). Thus I will be able to call C functions on my firmware from the PC with python (We'll need that for non-time-criticial unittesting of hardware). Nice, eh? ;-)

---

regarding the in-memory wrapper/caching:

In fact I see no use case for it apart from enforcing compatibility to 0.1.1. The usual use case is:

• The user want to bind his application against a fix version of the library
• He runs the binding generator to generate a python file from the librarys header file
• He adds the generated file to the project and includes it via regular import statement.

Did I miss possible other use cases? Tell me about the use case you mentioned (by the way, what do you mean with "proprietary headers"?).

---

regarding concept of implementation:

Without having digged into the details I think a "backend" should be as simple as writing templates in a template engine syntax. Do you have preferences regarding the template engine? I think it should fulfill the following requirements:

• small footprint, so that we can include as part of pyclibrary and avoid external dependencies (as you did with pyparsing)
• should be widely used. I do not know if there is a "most used" engine? do you?
• should have a syntax, that is also available in other languages. This will encourage non-python developers to write/work on plugins for their favorite language, since there are chances that they know the template engine syntax already.

Currently I favour Jinja2 due to the fact, that its syntax is also used for Liquid, which is used by GitHub Pages => better chances that a lot of people are familiar with the syntax. Furthermore my impression is, that it is very widespread?!? To get rid of the bloated jinja2 library, we could maybe write our own "micro-jinja2-implementation" later. This module then implements only features needed by our backends and would be very minimalistic.

Any better idea?

---

regarding distribution of workload:

As I mentioned already I would do the complete job if you are not interested in investing time. But of course I would be glad if we could share the work a little bit.

[MatthieuDartiailh]

regarding the new binding generator :

1. we can just start with smart binding but it seems to me that to get there we kind of need th basic one so I listed it first no big deal.
2. I completely agree.
3. This depends on the actual library being wrapped up as if you need to access often to it ctypes overhead may be an issue but this is very long term view.
4. we will see if we become really popular it might be a nice adventure.
5. Nice indeed reminds a bit of the this project https://github.com/m-labs/migen which focus on FPGA board.

---

regarding the in-memory wrapper/caching:

I am actually using pyclibrary to wrap a dll used to communicate with a digitalizer board and as I am not supposed to disclose the header files, I doubt I can disclose the wrapper.

For all other use cases you are right. Anyway when using it it should quite easy to check whether or not the wrapper has already been generated and if not generate it locally..

---

regarding concept of implementation:

For a not smart backend we can go simply with a template engine I guess. I am not familiar with any but I do know about Jinja2 which I think indeed quite popular. However for a smart one I think it nice to provide abstractions such as pointer creation, array creation, ... Those are basically the not implemented private method of clibrary object. For each kind of ffi some standard function could exist, that makes easier to switch from one ffi to another. (I am not yet sure of what will happen of ctypes vs cffi for python and I think that having a high-level abstraction not caring about which one you use is nice).

---

regarding distribution of workload:

As we go on I will try to give you a hand. I cannot make promises as I am trying to develop/maintain/contribute to far too many libraries in parallel of my work, but if we can draw a clean separation between two tasks I will be happy to work on one. One possibility would be for me to work on the preprocessor while you work on the parser perhaps. Could you open an issue with your ideas about that separation ?

[mrh1997]

I think we agree already about the rough concept ;-)

To summarize it:

• we start creating the binding generator for ctype-bindings. That means the resulting file is a pure python module that contains a lot of ctypes objects.
• The decision if we start pure or smart is not so important yet, as it is easy to add the functionality of the second one, if the first one is done. Before starting this issue I will look into the backend implementation. Then we can finish this part of the discussion.
• The template engine will be probably jinja2. But when starting with this issue, I will look into template engine options. Maybe I find another, better matching engine...
• I will do all the work. But If there is are issues that clearly can be separated, you might be interested in doing it. Lets check this out before starting.

Regarding our current plan, that you integrate the current c_model.py into the backend, while I integrate it into the parser: I think, you can cancel this job, as the backend will be completely replaced => your work would be for no purpose.

Regarding your special case for caching: it will be easily be possible to 'emulate' it simply by the following code in your application:

try:
    import binding_for_xyz
except ImportError:    # bindings were not generated yet
    import binding_generator
    binding_generator.translate(input='xyz.h', output='binding_for_xyz.py)
    import binding_for_xyz
else:
    import md5
    if binding_for.xyz.version != required_version  or  md5.new(open(
xyz.h').read()) != binding_for_xyz.header_file_chksum:
         binding_generator.translate(input='xyz.h', output='binding_for_xyz.py)
         reload(binding_for_xyz)

[mrh1997]

Unfortunately I found no Jinja2 compatible mini templating engine.

Titen (https://code.google.com/p/titen/) looks fairly promising, as it has only ~200 lines of code.

But I prefer to use Tempita (http://pythonpaste.org/tempita/) as it is still not too big (~1200 LOCs) and provides much more power.

[mrh1997]

Our project priorities moved somewhat. This means I will continue with pyclibrary in two or three month. Hope this is acceptable for you...

[MatthieuDartiailh]

I am fine with that. I am myself quite busy for now. By the way, are you set on tempita as a template engine ? If it is so, I can try to push to get the python 2/3 compatibility in and perhaps start some experimentations if I find some time. If for any reasons you cannot come back to this work please let me know.

[mrh1997]

I will come back to this work. But I will keep you informed if there would be further delays.

regarding tempita: I am very unhappy with all template engines as I detected they all have a big drawback: they do not allow codestyling as they do not know the destination languages AST. They are either doing simple text processing (without any code formatting) or are specialized to XML/HTML AST. This means if I want to output Python or C instead of HTML, the result is very ugly or even invalid (i.e. correct indentation in python is not a question of elegance but it is a must).

This is why I am playing around with my own template engine that works the following way:

• you have to specify AST classes for an arbitrary destination language
• you specify one (or more) code style(s) for this AST (= a formal description how the AST is transformed to text)
• To generate code, an AST has to be build and passed to my template engine together with a code style definition. The template engine will do the rest.

As I do not know if it really will work out as I image, I am currently creating a prototype. As soon as I have a prototype that I am happy with, I will come back to you to discuss the idea.

[MatthieuDartiailh]

Seems interesting. Keep me posted.

[MatthieuDartiailh]

Any progress on this ?

[mrh1997]

No. I am really sorry for this.

Actually our company got some very important projects, which will move this issues away for at least a couple of month. This means I had to it in my spare time or wait until next year.

I definitely will continue the job but probably at end of this year ;-( Is this a problem for you?

[MatthieuDartiailh]

For the time being it is not a problem but I may need to convince some people in the not so far future that using pyclibrary is a good idea and it is not easy when you expect to change the API on a short time scale. Actually I am also short on time, so I can easily understand you. Would you consider posting your experimental templating system on github so that I can give a look and perhaps try move a bit forward ?

[mrh1997]

What is "the not so far future", when you have to convince some people of pyclibrary? Can you give me at least two or three weeks to build the templating system far enough to allow s.b. else (i.e. you) to understand the idea?

[MatthieuDartiailh]

I can, I think, easily give you a month but I don't want to rush you or steal the templating project from you. My point is rather that if you make it available on github and provide me with some guidelines I may work on it and make pull request while you are busy. You will then be able to review my work which is less work than writing the code yourself. I will need to use pyclibrary for wrapping new dlls in my lab in something like a month and around the same time I will try to integrate it in another library I collaborate on. I don't need the final version then (as I do not think we can manage it) but I would like to say that efforts are being made and have myself a good idea of the future to try to make my code as easy to update as possible. Does that make sense to you ?

[mrh1997]

Sorry for leaving you so long without any response. But I didn't want to come back to you without showing you something. Unfortunately I needed much more time than expected to come to this point:

After playing around a lot with a lots of different ideas I think I have a very clean and lightweight approach. It is far from being complete (in fact I implemented only some elements for demonstration purpose). If you check out my "new-backend-api" branch you can have a look into it and give me your comments.

Here some important infos:

• I introduced two basic concepts: AST models and AST transformers (see pyclibrary/asts/astcore.py).
  • AST models are a tree of AstNode derived classes that are used to represent a AST. Currently we have two ASTs:
  • C subset (only declarations)
  • Python (full)
  • AST transformers are (parametrizable) objects, that translate from one AST to another. Currently we have two transformers:
  • C AST -> ctypes-bindings (=Python AST)
  • Python AST -> layouted text (Has to be reworked, as "layouted text" is not implemented as AST yet but as calls to CodeLayouter())
• For AST converters a class simply has to derive Transformer and register a method for every Input-AST class it wants to transform with Transformer.register decorator (see https://github.com/mrh1997/pyclibrary/blob/new-backend-api/pyclibrary/backends/ctypes_bindings.py).
• Transformers can be piped (as many as you want). This allows a very modular concept. Of course until now we need to stack only two, but it could be very useful to introduce "filters", which are doing some very specific transformation.
• The transformers are implemented as generators => very less resource consumption as the output can be processed "on the flow".
• The CTypesBindingsCreator.transform_clib_interface() ist not a registered transform method, as it gets not a AST node, but a CLibInterface object. But it works fairly smilar to transform methods()

If you are still interested in sharing the work: I think a good possibility to split the work would be the two Transformers: the rules for the C-AST->c-types-bindings transformer (pyclibrary/backends/ctypes_bindings.py) could be done by you, while I concentrate on reworking the Python-AST->layouted text transformer (pyclibrarey/backends/pep8_formatter.py). The second one has to be rewritten completely, as it uses the CodeLayouter(), which does not fit into the concept of ASTs and Transformers yet.

I am very curious for your thinking about the concept...

[MatthieuDartiailh]

I like the idea, really. But I have some questions :

• I am not 100% convinced by the need to introduce our own python ast. Could we not simply rely on the standard library ? (which would allow to use tool like this one for code generation https://github.com/berkerpeksag/astor)
• for the Python to code transform I am curious, have you considered using external tools (such as the one previously mentioned) ? Also one thing I would fine supper nice would be to have the ability to split the generated code in separate module (such as constants, library, utils (standard pyclibrary definition that we will need to copy)), how easy/hard do you thing that would be ?
• I like the idea of allowing to pipe transformers as it will allow the project to evolve more easily.

I am definitively interested in sharing the work, and I am fine with the separation you are proposing. Once we have discussed in a bit more details the points related to my questions I will be happy to start working on the ctypes transformer.

[mrh1997]

Honestly said, I played already around with the idea of using the integrated python AST module. But I didn't know "astor" so I decided against using the python AST. Now, after you told me of astor I started rethinking this idea.

But I still come to the conclusion that using our own AST is a better idea for two reasons:

• To make pyclibrary language agnostic (which is an important goal at least for my requirements) we will need a generic framework for AST to text convertion anyway. I would strongly prefer it, if the python way of AST to text conversion uses the same framework than for other languages.
• It is not mandatory any more, that the header file to python file translation step is done at runtime. Instead it can be done in a separate step during development where you might have a different python version. I.e. you might want to translate a header file to python 3 and python 2 syntax in a single step. But this is not possible when using the integrated AST, as it allows only to output code in the python version which runs the pyclibrary backend.

---

Regarding splitting the generated code I am not sure if I understood your intention. Did you want pyclibrary to output multiple .py files, each containing different parts of the header file (constants defs / func defs / structure defs)?

This case is doable fairly simple. We simply build not a sinlge Transformer but multiple transformers (one for constants, one for funcs, ...) and store the output of each of them in a separate .py file.

Ideally all transformers are the same class but parametrized differently (i.e. by a parameter called "output_only_constants", "output_only_funcs", ...). Alternatively we can build Filter Transformers that get a C AST and outputs a C AST with only constants/funcs/... These filters are then chained before python-AST-converter-Transformers

---

Great if we can split the work ;-) I would be very happy if we could do it the way as proposed...

[MatthieuDartiailh]

Your arguments for the AST makes sense (at least to me) and as I am not an expert on the matter (and as your implementation seems easy to maintain) I won't fight on it.

You got my idea about the separation. It is just that some project wrapping external dll do that kind of things to avoid having a single huge file cluttered with all the constants (defined as macros), and I thought people might appreciate the possibility to do the same using pyclibrary.

I will try to tackle the ctypes translation. I won't be able to work full time on it but I will try to keep progress steady if a bit slow. I will work on my repo under the new-backend-api branch (based on your current branch). Whenever you make significant progress you can open a PR against it. Being able to generate real code my also help me.

[MatthieuDartiailh]

I am starting to play with the transform idea, and I have a few questions :

• the python ast is incomplete (missing Raise, Import, Module etc). I guess it is because you had no time to complete it. Is it fine if I complete it ? We will need the Import and ImportFrom to be sure we are generating fully functional modules (ctypes imports). By having the Module node we allow the transformer to return several distinct modules which can be turned into multiple files.
• What is/was the rational for not including the evaluated value of a MacroVal in it but rather keeping it in a separate mapping ? I am asking because doing the evaluation the way you propose it in your proof on concept won't work if a macro if defined with respect to another (which happens). Our pre-processor takes care of that so storing the value into the ast would make sense.

[mrh1997]

Of course you can modify the AST (I coded it in 20 minutes without too much thinking about it). Actually there are more statements missing (i.e. "with"). My intention was to add them when required...

But please keep in mind that the AST should be abstract enough to hide python 2/3 specific parts. I.e. in python 3 you have to do a relative import like "from . import module_name", while in python 2.6 you have to write "import module_name". Thus I would introduce a single Import node. Of course this single Import node should be powerful enough to encode the semantics of both import variants. If we follow these rule, your C->Python Transformer has not to worry about:

1. if python 2 or 3 code should be emitted
2. what conventions for imports shall be used These details are handled by my code emitter transformer instead, where you can parametrize the python version and coding styles like import convention.

---

Regarding your idea of having a module node: I think this will reduce the modularity of the transformer concept. My idea (which I don't know yet if it works out) is to work with very atomic transformers. Depending on your use case you simply have to change the pipelines you are building from these transformers then.

I.e. we could have a transformer for constants and one for struct/union defs. Then you simply pipe each of them with a separate code emitter transformer to get separate modules. For those who want them to be outputted into a single file, we could provide a (generic) merge-transformer. This merge transformer can be used to merge both transformers into a single AST...

How do you think about this?

---

Regarding MacroVal: you are right, I didn't consider that.

The Problem is, that the macro def is a string of C code (like "A+B"), which cannot be converted to a python AST (we would require a full C parser, which also supports AST nodes for expressions like '+').

My proposal would be to replace the macro references used within a macro definition and THEN evaluate the result.

To make the result more readable we could add the original string as comment to the AST. Then #define C A+B will be transformed to C=4 # A+B (given A is 1 and B is 3). To get this we would require a 'comment' field for the generic AstNode.

What are your ideas?

[MatthieuDartiailh]

Regarding imports I think it would be better to make the ast closer to Python3 and let the translator handle the change (note that I do not support python 2.6 and that from . import foo is correct in Python 2.7). Creating an ast that differ too much from the real python one will make our life harder.

---

The thing that worry me is that if we do not have a Module node we won't know where to put the imports. From your description I guess that we would actually have some sort of super structure taking care of chaining the transform and such, and that such a construct could inject the generated code into some kind of templates. This may indeed make sense as we will have to package some utility functions and base classes that are more easily written as text that using our own ast. We can keep that for later.

---

My point for macro val was actually more straightforward. All values are evaluated during the pre-processing (which may be debatable), but is easy to do because we have the full context needed to do the evaluation. And we actually store them. We could include inside the ast node the value under a 'value' member. But actually coding the types transformation I realized the transformer need to store a ref to the clib because it contains informations without which we cannot resolve the custom types. So I went with storing the clib into a '_clib' attribute (in transform_clib) that can then be accessed by all transform that need to dive into it for some infos.

I will try to go on with the basic transformations and see what comes out.

[mrh1997]

Probably you are right. Lets stick to the python 3 AST. If we are lucky it is abstract enough to transform it to python 2 source code. If not it should not be a big deal to adapt it slightly...

---

After sleeping a night this came to my mind, too. So lets start with your module node proposal as it is less experimental and will be implementable without unexpected problems. After we have a running system we can think about refactoring it (if we see it could make sense).

---

I do not like cyclic structures as they can cause trouble in python (i.e. will not be garbage collected as soon as one of member objects has a del method). But if I understood you correctly you added this c_lib reference to ValMacro which results in a cyclic reference...

To address this kind of problem I added the possiblity to provide contexts to transformers (see 'ctx' in https://github.com/mrh1997/pyclibrary/blob/new-backend-api/pyclibrary/asts/astcore.py#L145). This way the CLibInterface tranformer function could provide the clib to the ValMacro transformer function.

---

Another problem came to my mind: When working with ValMacro we could do eval. But how about FnMacro? Here we cannot simply run a eval, as we do not have all parameters during compile time.

The only idea to solve this issue is a ugly workaround: We introduce a special Python AST Node "UnstructuredExpression" which simply contains a string of python code. Your transformer will emit this node when coming to macros. My transformer will route it directly into the sourcecode without further processing.

Any different idea?

[MatthieuDartiailh]

Actually I added the clib to the Transformer as otherwise we would need to always pass it as a context as we cannot know when we will need it. So there is no cyclic references.

ValMacro needs to be exposed on the python side as those are often used as arguments to the function. However FnMacro does not need to be imported as they are never needed by consumer code (correct me if this is not true).

[mrh1997]

As in C FnMacros are often used as replacement for functions (i.e. to ensure legacy code compatibility or for performance reasons) I think we should not ignore them.

But I agree that this is far less needed as the rest => lets ignore them in a first step...

[MatthieuDartiailh]

Of course Macros can be used for that, but I don't think that the dll can export it.

[MatthieuDartiailh]

Hi, I took some time to rewrite the python ast. I chose to follow closely the actual python 3 AST as the translator can always (nearly) find a way to rewrite it in a compatible fashion. Also I was not a big fan of your custom init added by the metaclass, so I moved to a kwargs only solution and made possible to easily specify default values for some slots. Feel free to comment.

[mrh1997]

Fine. Working with kwargs is much more elegant when using the ast.

[mrh1997]

after digging a little bit more into the code, I have some questions:

• What exactly is the reason for the "Context" class? Isn't it redundant? I.e. in Assign([Name('var_name', Store())], [Int(1)]) 'Store()' is not really needed, as 'Assign' implies already that 'var_name' has to be stored (not loaded).
• In my understanding all classes have to be derived (directly or indirectly) from statement or expression (in fact expression have to be derive from statement, too, as it is a statement). But a lot of classes derive from AstNode. This is important for the rules of the transformer-concept, as they rely on semanticially correctly derived classes (i.e. you could create a generic rule for a common base class and an extra one for each subclass you want to handle special).
• Why do we need 'Expr'?

[MatthieuDartiailh]

I will try to answer the best I can.

• I am actually not completely sure of what the meaning of the Context is in the Python ast. I included it to be as complete as possible and allow potential conversion to the standard python ast (but I don't have a specific use in mind).
• Classes that derive from AstNode are some kind of helpers class that are not defined in the grammar as expressions nor statement (see https://docs.python.org/3.5/library/ast.html) that's why I made them subclasses of the base AstNode class.
• Expr is for example used in function body for a docstring, this is a bit artificial but as I am not very familiar with ast stuff I tried to stick the Python AST.

I am not claiming that my implementation is perfect (or better than yours). We can still change it (and change the inheritance relations). I just like the idea to stating close the real ast.

[mrh1997]

I agree that following the Python 3 AST is a nice idea. But I still think that we should concentrate on our requirements. And part of this requirements are a clear inheritance relation, as the transformers are utilizing the mro of each ast object to identify the correct transformation rule. If it is OK for you I will adapt inheritance slightly when implementing the backend and I see that it simplifies my job.

[MatthieuDartiailh]

Feel free to experiment it is important we have a solid AST to build upon.

[mrh1997]

Bad News: your company decided to check if libclang could match our needs (instead of pyclibrary). If this works out, I will not be able to spend time on this project any more... Sorry for leaving the project in an incomplete state...