pygccxml.utils.xml_generator not always set

[mamoll]

When trying to regenerate python bindings and using the cache file mechanism I run into the following problem:

INFO Loading cache file "/Users/mmoll/build/omplapp/Release/pyplusplus_util.cache".
INFO Parsing source file "bindings/util.h" ... 
Traceback (most recent call last):
  File "/Users/mmoll/omplapp/ompl/py-bindings/generate_bindings.py", line 803, in <module>
    globals()['ompl_'+module+'_generator_t']()
  File "/Users/mmoll/omplapp/ompl/py-bindings/generate_bindings.py", line 737, in __init__
    code_generator_t.__init__(self, 'util', None, replacement, 1)
  File "/Users/mmoll/omplapp/ompl/py-bindings/ompl/bindings_generator.py", line 157, in __init__
    indexing_suite_version = indexing_suite_version )
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pyplusplus/module_builder/boost_python_builder.py", line 95, in __init__
    , indexing_suite_version)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pyplusplus/module_builder/boost_python_builder.py", line 138, in __parse_declarations
    decls = reader.read_files( files, compilation_mode )
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pygccxml/parser/project_reader.py", line 249, in read_files
    return self.__parse_file_by_file(files)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pygccxml/parser/project_reader.py", line 315, in __parse_file_by_file
    reader.join_declarations(ns)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pygccxml/parser/source_reader.py", line 488, in join_declarations
    self.join_declarations(ns)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pygccxml/parser/source_reader.py", line 485, in join_declarations
    self._join_namespaces(declref)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pygccxml/parser/source_reader.py", line 507, in _join_namespaces
    if decl not in joined_decls[decl._name]:
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pygccxml/declarations/calldef.py", line 249, in __eq__
    if "GCC" in utils.xml_generator:
TypeError: argument of type 'NoneType' is not utterable

[iMichka]

I think I know what is going on, I already encountered this problem but I did not think it would cause any trouble.

Maybe my approach with this global utils.xml_generator is a little bit fragile. I'll have a look soon.

[mamoll]

So far, I have also needed to make this (unrelated) change:

--- pygccxml/declarations/calldef.py    2015-11-15 11:33:22.000000000 -0600
+++ pygccxml/declarations/calldef.py    2015-11-24 13:56:26.000000000 -0600
@@ -745,8 +745,11 @@
         arg = args[0]
         if not type_traits.is_reference(arg.type):
             return False
+        try:
         if not type_traits.is_const(arg.type.base):
             return False
+        except:
+            return False
         unaliased = type_traits.remove_alias(arg.type.base)
         # unaliased now refers to const_t instance
         if not isinstance(unaliased.base, cpptypes.declarated_t):

It turns out that arg.type can be of type declarated_t which does not have the base property.

[iMichka]

Hi

I looked at the second problem with the copy_constructor. I was not able to reproduce it. I can have arguments which have arg.type.base of type declarated_t, but not arg.type of type declarated_t. Could you provide an small c++ code example, so that I can write a unit test for this case ?

[mamoll]

I'm working on narrowing down the second problem, but it's tricky. So far, I have found that pygccxml/py++ chokes on boost::tuples::access_traits::parameter_type.

[mamoll]

Here is a minimal example for the second problem. Header file test.h:

#include <boost/tuple/tuple.hpp>

class tuple1 : public boost::tuple<unsigned long>
{
public:
    tuple1(boost::tuples::access_traits<unsigned long>::parameter_type t0)
        : boost::tuple<unsigned long>(t0)
    {
    }
};

Python binding generation code:

from pyplusplus import module_builder
from pygccxml import parser

# fails with this setting:
xml_generator_config = parser.xml_generator_configuration_t(xml_generator="castxml",
    xml_generator_path="/opt/local/bin/castxml", include_paths=['/opt/local/include'])
# succeeds with this setting:
# xml_generator_config = parser.xml_generator_configuration_t(xml_generator="gccxml",
#     xml_generator_path="/opt/local/bin/gccxml", include_paths=['/opt/local/include'])
mb = module_builder.module_builder_t(files=['test.h'], xml_generator_config = xml_generator_config)
mb.build_code_creator( module_name='tupletest' )
mb.write_module('tupletest.cpp' )

[ghost]

Hello @mamoll I tried your python binding generation code which is something like this:

from pygccxml import parser
from pygccxml import utils
from pyplusplus import module_builder

# Find out the xml generator (gccxml or castxml)
generator_path, generator_name = utils.find_xml_generator(name='castxml')

# Configure the xml generator
 xml_generator_config = parser.xml_generator_configuration_t(
                   xml_generator=generator_name,
               xml_generator_path=generator_path, 
           compiler="gcc")

mb = module_builder.module_builder_t(files=['hello_world.hpp'],   
                                   xml_generator_config=xml_generator_config) 

And, I got following error message:

TypeError: __init__() got an unexpected keyword argument 'xml_generator_config'

I am wondering how it worked for you.

[mamoll]

@jamir414, make sure you get pyplusplus from here: https://bitbucket.org/ompl/pyplusplus. I have been trying to update the PyPI entry, but to no avail (http://sourceforge.net/p/pypi/support-requests/566/).

[ghost]

@mamoll Actually, I did got pyplusplus from that link ( https://bitbucket.org/ompl/pyplusplus/downloads ) and pygccxml from https://github.com/gccxml/pygccxml

[mamoll]

@jamir414, did you download the whole repo or one of the prepackaged releases? Only the former has support for the newest pygccxml.

[ghost]

@mamoll I downloaded only the latest release. I will try again after downloading the whole repo like you said. Thank you.

[ghost]

@mamoll It worked without any problem. Thank you.

[iMichka]

I wrote a simplified test to reproduce the error for the missing arg.base, 74ffdde4cfb4731d27a7624b8dd6019330868812

It seems that this is also broken with gccxml ...

Checking for the copy constructor works with:

class test {
      public:
           test(const int & t0) {}
};

But fails with:

typedef const int & a;
class test {
       public:
           test(a t0) {}
};

In the second case (like in your boost example), a is of type declarated_t. But I think it should be of type reference_t (the first case is of type reference_t). I will continue to investigate but the cause is probably somewhere between reading the file and populating the declarations tree.

I would prefer to not implement the try/catch block for the moment without understanding what is going on. Not sure if there is a difference in C++ between these two ways of writing the constructor. Because I would still expect the second one to be a copy constructor (so returning False is not correct).

For the second problem with the caching, I was not able to reproduce it. I tried playing around with the cache. Would it be possible to get also a small example for this problem ?

[mamoll]

I am sorry I haven't had time to create a small example. If you're interested, you can try looking at the errors when building OMPL, the project that depends on pyplusplus and pygccxml:

git clone ssh://hg@bitbucket.org/ompl/ompl
mkdir -p build/Release
cd build/Release
cmake ../..
make -j 4 update_bindings && make -j 4 make

[iMichka]

The copy constructor problem is now fixed. I create a 1.7.2 branch which will contain the fix.

I tried to build ompl, and had no problems. The update_bindings target was not there, but OMPL_BUILD_PYBINDINGS was set to on so I guess the python bindings were built. I edited some c++ files to re-trigger a build (and hopefully a re-read of some cached files), but could not reproduce the error.

[mamoll]

The update_bindings target is defined automatically if castxml/gccxml, pygccml, and pyplusplus are installed. (See https://bitbucket.org/ompl/ompl/src/tip/CMakeModules/Findxmlgenerator.cmake and https://bitbucket.org/ompl/ompl/src/tip/CMakeModules/PythonBindingsUtils.cmake for details.) I'll test the 1.7.2 branch shortly. Which version of python do you use?

[iMichka]

It was python 2.7

[mamoll]

@iMichka, you most likely did not build the python bindings for OMPL if cmake didn't detect castxml/gccxml, pygccml, and pyplusplus. The good news is that with the hotfix/v1.7.2 branch the copy constructor error is now fixed. The bad news is that running "make update_bindings" twice still fails in the same way as described at the top of the page. I am using python 3.5 now. It's a bit of a pain to recompile boost.python with python2.7, but if you think that might matter I'm willing to try it. If you are sure you have castxml, pygccxml, and pyplusplus installed, perhaps you can try configuring OMPL like so:

cmake -DPYTHON_EXEC=/opt/local/bin/python3.5 \
   -DPY_PYGCCXML:STRING=/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pygccxml \
   -DPY_PYPLUSPLUS:STRING=/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyplusplus \
   -DXMLGENERATORPATH:FILEPATH=/opt/local/bin/castxml

(Obviously, you'd substitute your python path and the right module paths.)

[iMichka]

Ok I had pyplusplus not installed. I got the bindings to build, and also got the error on the second run. I'll look at it now and try to write some minimal test first. I'll probably debug this on python3, the stack trace is much nicer there :)

[iMichka]

I fixed the problem, should be good now.

There may be other problems while compiling ompl, please open a new bug if it is the case (I think my build failed but I just updated boost on OS X so this may have broken it, I did not investigate the new errors).

Please close this bug once you tested the v1.7.2 branch. I will then make a new release to get these fixes out. Other bugs will then be fixed in v1.7.3.

[mamoll]

Yes, the original problem is fixed. Thank you! Feel free to file a bug or send me a message with the ompl build problem you observed.