search

carla-simulator / map

AD Map access library
MIT License
96 stars 43 forks source link

Python binding building incomplete on Ubuntu18.04 #18

Closed hibetterheyj closed 4 years ago

hibetterheyj commented 4 years ago

OS: Ubuntu18.04

Got error when building as follows:

➜  ad_map_access git:(master) cmake ../../ad_map_access -DCMAKE_INSTALL_PREFIX=../../install -DBUILD_PYTHON_BINDING=ON

The terminal is as follows:

-- Found Boost: /usr/include (found version "1.65.1") found components:  program_options filesystem system 
-- Found Boost: /usr/include (found version "1.65.1")  
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.17", minimum required is "2") 
CMake Warning at /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.15/Modules/FindBoost.cmake:2003 (message):
  No header defined for python-py27; skipping header check
Call Stack (most recent call first):
  /home/heyujie/Projects/map/cmake/python-binding.cmake:55 (find_package)
  python/CMakeLists.txt:29 (find_python_binding_packages)

-- Found Boost: /usr/include (found version "1.65.1") found components:  python-py27 
-- PythonLib 2 found. /usr/include/python2.7, /usr/lib/x86_64-linux-gnu/libpython2.7.so
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.7m.so (found suitable version "3.7.8", minimum required is "3") 
CMake Warning at /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.15/Modules/FindBoost.cmake:2003 (message):
  No header defined for python-py37; skipping header check
Call Stack (most recent call first):
  /home/heyujie/Projects/map/cmake/python-binding.cmake:84 (find_package)
  python/CMakeLists.txt:29 (find_python_binding_packages)

CMake Error at /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.15/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find Boost (missing: python-py37) (found version "1.65.1")
Call Stack (most recent call first):
  /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.15/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.15/Modules/FindBoost.cmake:2161 (find_package_handle_standard_args)
  /home/heyujie/Projects/map/cmake/python-binding.cmake:84 (find_package)
  python/CMakeLists.txt:29 (find_python_binding_packages)

-- Configuring incomplete, errors occurred!
See also "/home/heyujie/Projects/map/build/ad_map_access/CMakeFiles/CMakeOutput.log".
See also "/home/heyujie/Projects/map/build/ad_map_access/CMakeFiles/CMakeError.log".

BTW, any ideas to specify the Python version (preference: Python3.6)

Thanks in advance!

berndgassmann commented 4 years ago

Hi,

in general it should compile on Ubuntu 18.04. Especially because the CI is also building it under that configuration: https://travis-ci.com/github/carla-simulator/map/jobs/365357839 Maybe you try to install the packages listed in Travis configfile: https://github.com/carla-simulator/map/blob/master/.travis.yml

Since the error says that boost python is not found, is libboost-python-dev installed on your system?

If you want that a specific version to be used ... you might want to edit find_python_binding_packages() function in here: https://github.com/carla-simulator/map/blob/master/cmake/python-binding.cmake and set the paths manually there, or search for what options the find_package(PythonLibs 3) might have to select a specific version...

Regards, Bernd.

hibetterheyj commented 4 years ago

I have installed the libboost-python-dev with the version of 1.65.1.0ubuntu1

berndgassmann commented 4 years ago

Hi, did you find a solution now? When reading your output what is somehow strange is that python 3.7 and python 3.6 seem to be mixed up some how:

In the travis log here: https://travis-ci.com/github/carla-simulator/map/jobs/365357839 There seems to be installed python 3.6.

I've tried on my machine to edit the line I mentioned https://github.com/carla-simulator/map/blob/master/cmake/python-binding.cmake#L76

When I also request the Minor version, a find_package(PythonLibs 3.6) fails to find the python 3 package and find_package(PythonLibs 3.5) works for me on Ubuntu 16.04. (where indeed python 3.5 is installed ...)

So I believe if you would change the line on your machine to find_package(PythonLibs 3.6) that should enable the compilation on your machine without the need of Deinstallation of the python3.7.

I could imagine, that the root cause is that the boost 1.65.1 installed in the system was not compiled for python 3.7; but rather for the default 3.6 python version. I remember a similar issue reported here: https://github.com/intel/ad-rss-lib/issues/74 where boost version and python version didn't match each other.

Another source of problems could come with the cmake version (which provides the FindBoost, etc. modules ....) [there I had to fight with when getting newer boost version to work on an older machine .... some time ago]. So whenever you change something from the system maintained versions you might run into such issues... which are hard to debug to the ground....

felipecode commented 4 years ago

I am also having problems with the python biding installation

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeGeneratedPointTypes.hpp

  /usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the
  imp module is deprecated in favour of importlib; see the module's
  documentation for alternative uses

    import fnmatch, glob, traceback, errno, sys, atexit, locale, imp

  Traceback (most recent call last):

    File "generate_python_lib.py", line 87, in <module>
      main()
    File "generate_python_lib.py", line 55, in main
      ignore_files=ignore_files)
    File "/home/felipe/map/build/ad_map_access/python/python_wrapper_helper.py", line 97, in generate_python_wrapper
      indexing_suite_version=2)
    File "/home/felipe/.local/lib/python3.6/site-packages/pyplusplus/module_builder/boost_python_builder.py", line 106, in __init__
      , indexing_suite_version)
    File "/home/felipe/.local/lib/python3.6/site-packages/pyplusplus/module_builder/boost_python_builder.py", line 149, in __parse_declarations
      decls = reader.read_files( files, compilation_mode )
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/project_reader.py", line 255, in read_files
      return self.__parse_all_at_once(files)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/project_reader.py", line 356, in __parse_all_at_once
      return self.read_string(''.join(header_content))
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/project_reader.py", line 369, in read_string
      decls = reader.read_string(content)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/source_reader.py", line 373, in read_string
      decls = self.read_file(header_file)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/source_reader.py", line 299, in read_file
      return self.read_cpp_source_file(source_file)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/source_reader.py", line 318, in read_cpp_source_file
      xml_file = self.create_xml_file(ffname)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/source_reader.py", line 267, in create_xml_file
      ": %s status:%s" % (msg, exit_status))

  RuntimeError: Error occurred while running CASTXML: status:1

I have tried several versions of the castxml , pygccxml and pyplusplus, also different version for python3. My Boost version: 1.65.1 and I am on ubuntu 18.04.

I checked the compatible python version with the boost version and I am using the correct one py36

find /usr/lib -iname 'libboost_python.so'

/usr/lib/x86_64-linux-gnu/libboost_python-py36.so /usr/lib/x86_64-linux-gnu/libboost_python3-py36.so /usr/lib/x86_64-linux-gnu/libboost_python.so /usr/lib/x86_64-linux-gnu/libboost_python3-py36.so.1.65.1 /usr/lib/x86_64-linux-gnu/libboost_python3.so /usr/lib/x86_64-linux-gnu/libboost_python-py27.so /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.65.1

berndgassmann commented 4 years ago

Hmm,

so I got now hand on an Ubuntu18.04 and compiled it there. @felipecode: The code generation worked, the system used python2.7, and in my site-packages I've found pygccxml-1.9.1.dist-info, pyplusplus-1.8.1.egg-info. At least that seems to be a valid combination.

I'll try now how it behaves if I update to newer versions ... maybe I can reproduce the issue ...

berndgassmann commented 4 years ago

Update of the two packages installed versions: pygccxml-2.0.1 and pyplusplus-1.8.3 and it still compiles.

Now trying to enforce usage of python3 by changing the command in cmake/python-binding.cmake from python to explicitly python3. Furthermore, called pip3 install --user pyplusplus.

The generation of the code gives a lot warnings, but it doesn't crash. And the python tests work in the end.

Auto-generating python code from C++ with pyplusplus is not supporting all (and cannot in the end since C++ with types, overloading and virtual functions makes it hard). That's the reason why some of the files are explicitly excluded. Furthermore, I know that code generation has especially problems with the physics operator overloads, so that not all things are working from within python properly (calling the 'wrong' a = b *c leads to endless loops), but at least many features of the C++ library can be used from python side and are working.

By the way that is the output of 'apt search castxml' on that machine. castxml/bionic,bionic,now 0.1+git20170823-1 amd64 [installed]

berndgassmann commented 4 years ago

Hmm,

maybe also these two outputs. What is strange, that I don't see the warning that you see from your system:

/usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import fnmatch, glob, traceback, errno, sys, atexit, locale, imp

Per default the cmake calls just 'python' to trigger the creation of the python c++ code. What does your call to this bring?

$ python --version Python 2.7.17 $ python3 --version Python 3.6.9

felipecode commented 4 years ago

I get python2 version 2.7.17 and python3 version 3.6.9, same as yours. I tried castxml from apt-get and also installing a newer version directly from source. Both give the same error. Currently it is on version 0.3.20200427-g0a96f62

I changed the pythonbinding.cmake to call "python3". But for python2 I also got the same error

My pygccxml is version 2.0.1 and pyplusplus is 1.8.3

I think there might be more information. Here is the full cmake output.


-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1") 
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   program_options
--   filesystem
--   system
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Boost version: 1.65.1
-- Boost version: 1.65.1
The people
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.6m.so (found suitable version "3.6.9", minimum required is "3.6") 
CMake Warning at /usr/share/cmake-3.10/Modules/FindBoost.cmake:1626 (message):
  No header defined for python-py36; skipping header check
Call Stack (most recent call first):
  /home/felipe/map/cmake/python-binding.cmake:57 (find_package)
  python/CMakeLists.txt:29 (find_python_binding_packages)

-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   python-py36
PYTHOn 3 Almost DONE
--   PYTHON3: Includes: /usr/include/python3.6m; Libs: Boost::python-py36;/usr/lib/x86_64-linux-gnu/libpython3.6m.so
PYTHOn 3 DONE
-- Generating python binding source code at /home/felipe/map/build/ad_map_access/python
CMake Warning at /home/felipe/map/cmake/python-binding.cmake:30 (message):
  Generating result: 1
Call Stack (most recent call first):
  python/CMakeLists.txt:33 (generate_python_binding_source_code)

CMake Warning at /home/felipe/map/cmake/python-binding.cmake:31 (message):
  Generator output: In file included from /tmp/tmpsmw2qyrv.h:3:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/landmark/Operation.hpp:12:

  /home/felipe/map/install/include/ad/map/landmark/LandmarkOperation.hpp:32:58:
  error: redefinition of default argument

  inline bool isValid(Landmark const &landmark, bool const logErrors = true)

                                                           ^           ~~~~

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/landmark/LandmarkOperation.hpp:32:58:
  note: previous definition is here

  inline bool isValid(Landmark const &landmark, bool const logErrors = true)

                                                           ^           ~~~~

  In file included from /tmp/tmpsmw2qyrv.h:3:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/landmark/Operation.hpp:12:

  /home/felipe/map/install/include/ad/map/landmark/LandmarkOperation.hpp:42:62:
  error: redefinition of default argument

  inline bool isValid(LandmarkId const &landmarkId, bool const logErrors =
  true)

                                                               ^           ~~~~

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/landmark/LandmarkOperation.hpp:42:62:
  note: previous definition is here

  inline bool isValid(LandmarkId const &landmarkId, bool const logErrors =
  true)

                                                               ^           ~~~~

  In file included from /tmp/tmpsmw2qyrv.h:3:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/landmark/Operation.hpp:12:

  /home/felipe/map/install/include/ad/map/landmark/LandmarkOperation.hpp:108:23:
  error: redefinition of 'getVisibleTrafficLights'

  inline LandmarkIdList getVisibleTrafficLights(lane::LaneId const &laneId)

                        ^

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/landmark/LandmarkOperation.hpp:108:23:
  note: previous definition is here

  inline LandmarkIdList getVisibleTrafficLights(lane::LaneId const &laneId)

                        ^

  In file included from /tmp/tmpsmw2qyrv.h:4:

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/landmark/LandmarkIdSet.hpp:35:22:
  error: redefinition of 'operator<<'

  inline std::ostream &operator<<(std::ostream &os,
  ::ad::map::landmark::LandmarkIdSet const &landmarkIdSet)

                       ^

  /home/felipe/map/install/include/ad/map/landmark/LandmarkIdSet.hpp:35:22:
  note: previous definition is here

  inline std::ostream &operator<<(std::ostream &os,
  ::ad::map::landmark::LandmarkIdSet const &landmarkIdSet)

                       ^

  In file included from /tmp/tmpsmw2qyrv.h:4:

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/landmark/LandmarkIdSet.hpp:50:27:
  error: redefinition of 'to_string'

  static inline std::string to_string(::ad::map::landmark::LandmarkIdSet
  const &landmarkIdSet)

                            ^

  /home/felipe/map/install/include/ad/map/landmark/LandmarkIdSet.hpp:50:27:
  note: previous definition is here

  static inline std::string to_string(::ad::map::landmark::LandmarkIdSet
  const &landmarkIdSet)

                            ^

  In file included from /tmp/tmpsmw2qyrv.h:9:

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/ECEFCoordinateOperation.hpp:19:41:
  error: redefinition of 'operator*'

  inline ::ad::map::point::ECEFCoordinate
  operator*(::ad::map::point::ECEFCoordinate const &a,

                                          ^

  /home/felipe/map/install/include/ad/map/point/ECEFCoordinateOperation.hpp:19:41:
  note: previous definition is here

  inline ::ad::map::point::ECEFCoordinate
  operator*(::ad::map::point::ECEFCoordinate const &a,

                                          ^

  In file included from /tmp/tmpsmw2qyrv.h:9:

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/ECEFCoordinateOperation.hpp:30:41:
  error: redefinition of 'operator*'

  inline ::ad::map::point::ECEFCoordinate operator*(::ad::physics::Distance
  const &b,

                                          ^

  /home/felipe/map/install/include/ad/map/point/ECEFCoordinateOperation.hpp:30:41:
  note: previous definition is here

  inline ::ad::map::point::ECEFCoordinate operator*(::ad::physics::Distance
  const &b,

                                          ^

  In file included from /tmp/tmpsmw2qyrv.h:9:

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/ECEFCoordinateOperation.hpp:41:41:
  error: redefinition of 'operator/'

  inline ::ad::map::point::ECEFCoordinate
  operator/(::ad::map::point::ECEFCoordinate const &a,

                                          ^

  /home/felipe/map/install/include/ad/map/point/ECEFCoordinateOperation.hpp:41:41:
  note: previous definition is here

  inline ::ad::map::point::ECEFCoordinate
  operator/(::ad::map::point::ECEFCoordinate const &a,

                                          ^

  In file included from /tmp/tmpsmw2qyrv.h:9:

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/ECEFCoordinateOperation.hpp:52:41:
  error: redefinition of 'operator*'

  inline ::ad::map::point::ECEFCoordinate
  operator*(::ad::map::point::ECEFCoordinate const &a,

                                          ^

  /home/felipe/map/install/include/ad/map/point/ECEFCoordinateOperation.hpp:52:41:
  note: previous definition is here

  inline ::ad::map::point::ECEFCoordinate
  operator*(::ad::map::point::ECEFCoordinate const &a,

                                          ^

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:11:

  /home/felipe/map/install/include/ad/map/point/BoundingSphereOperation.hpp:23:26:
  error: redefinition of 'distance'

  inline physics::Distance distance(BoundingSphere const &left,
  BoundingSphere const &right)

                           ^

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/BoundingSphereOperation.hpp:23:26:
  note: previous definition is here

  inline physics::Distance distance(BoundingSphere const &left,
  BoundingSphere const &right)

                           ^

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:15:

  /home/felipe/map/install/include/ad/map/point/GeoOperation.hpp:33:55:
  error: redefinition of default argument

  inline bool isValid(GeoPoint const &point, bool const logErrors = true)

                                                        ^           ~~~~

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/GeoOperation.hpp:33:55:
  note: previous definition is here

  inline bool isValid(GeoPoint const &point, bool const logErrors = true)

                                                        ^           ~~~~

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:15:

  /home/felipe/map/install/include/ad/map/point/GeoOperation.hpp:43:53:
  error: redefinition of default argument

  inline bool isValid(GeoEdge const &edge, bool const logErrors = true)

                                                      ^           ~~~~

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/GeoOperation.hpp:43:53:
  note: previous definition is here

  inline bool isValid(GeoEdge const &edge, bool const logErrors = true)

                                                      ^           ~~~~

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:15:

  /home/felipe/map/install/include/ad/map/point/GeoOperation.hpp:53:15:
  error: redefinition of 'degree2radians'

  inline double degree2radians(double degree)

                ^

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/GeoOperation.hpp:53:15:
  note: previous definition is here

  inline double degree2radians(double degree)

                ^

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:15:

  /home/felipe/map/install/include/ad/map/point/GeoOperation.hpp:63:15:
  error: redefinition of 'radians2degree'

  inline double radians2degree(double radians)

                ^

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/GeoOperation.hpp:63:15:
  note: previous definition is here

  inline double radians2degree(double radians)

                ^

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:15:

  /home/felipe/map/install/include/ad/map/point/GeoOperation.hpp:73:15:
  error: redefinition of 'toRadians'

  inline double toRadians(Latitude const &latitude)

                ^

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/GeoOperation.hpp:73:15:
  note: previous definition is here

  inline double toRadians(Latitude const &latitude)

                ^

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:15:

  /home/felipe/map/install/include/ad/map/point/GeoOperation.hpp:83:15:
  error: redefinition of 'toRadians'

  inline double toRadians(Longitude const &longitude)

                ^

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/GeoOperation.hpp:83:15:
  note: previous definition is here

  inline double toRadians(Longitude const &longitude)

                ^

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:15:

  /home/felipe/map/install/include/ad/map/point/GeoOperation.hpp:95:17:
  error: redefinition of 'createGeoPoint'

  inline GeoPoint createGeoPoint(Longitude const longitude, Latitude const
  latitude, Altitude const altitude)

                  ^

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/GeoOperation.hpp:95:17:
  note: previous definition is here

  inline GeoPoint createGeoPoint(Longitude const longitude, Latitude const
  latitude, Altitude const altitude)

                  ^

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:15:

  /home/felipe/map/install/include/ad/map/point/GeoOperation.hpp:162:29:
  error: redefinition of 'vectorExtrapolate'

  template <> inline GeoPoint vectorExtrapolate(GeoPoint const &a, GeoPoint
  const &b, double const &scalar)

                              ^

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/GeoOperation.hpp:162:29:
  note: previous definition is here

  template <> inline GeoPoint vectorExtrapolate(GeoPoint const &a, GeoPoint
  const &b, double const &scalar)

                              ^

  In file included from /tmp/tmpsmw2qyrv.h:10:

  In file included from
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Operation.hpp:19:

  /home/felipe/map/install/include/ad/map/point/Transform.hpp:25:18: error:
  redefinition of 'toECEF'

  inline ECEFPoint toECEF(GeoPoint const &point)

                   ^

  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/Transform.hpp:25:18:
  note: previous definition is here

  inline ECEFPoint toECEF(GeoPoint const &point)

                   ^

  fatal error: too many errors emitted, stopping now [-ferror-limit=]

  20 errors generated.

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/access/Store.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/access/Factory.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/route/RouteExpander.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/route/RoutePrediction.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/route/Route.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/route/RouteAStar.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/opendrive/AdMapFactory.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeGeneratedPhysicsTypes.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/IChecksum.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeGeneratedRestrictionTypes.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/StorageFile.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeableMagic.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeGeneratedAccessTypes.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/IStorage.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializerFileCRC32.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/ISerializer.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeGeneratedLaneTypes.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeGeneratedBasicTypes.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeGeneratedTypes.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeGeneratedLandmarkTypes.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/ChecksumCRC32.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/Serializer.hpp

  Skipping file:
  /home/felipe/map/ad_map_access/python/../impl/include/ad/map/serialize/SerializeGeneratedPointTypes.hpp

  /usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the
  imp module is deprecated in favour of importlib; see the module's
  documentation for alternative uses

    import fnmatch, glob, traceback, errno, sys, atexit, locale, imp

  Traceback (most recent call last):

    File "generate_python_lib.py", line 87, in <module>
      main()
    File "generate_python_lib.py", line 55, in main
      ignore_files=ignore_files)
    File "/home/felipe/map/build/ad_map_access/python/python_wrapper_helper.py", line 97, in generate_python_wrapper
      indexing_suite_version=2)
    File "/home/felipe/.local/lib/python3.6/site-packages/pyplusplus/module_builder/boost_python_builder.py", line 111, in __init__
      , indexing_suite_version)
    File "/home/felipe/.local/lib/python3.6/site-packages/pyplusplus/module_builder/boost_python_builder.py", line 154, in __parse_declarations
      decls = reader.read_files( files, compilation_mode )
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/project_reader.py", line 255, in read_files
      return self.__parse_all_at_once(files)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/project_reader.py", line 356, in __parse_all_at_once
      return self.read_string(''.join(header_content))
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/project_reader.py", line 369, in read_string
      decls = reader.read_string(content)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/source_reader.py", line 373, in read_string
      decls = self.read_file(header_file)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/source_reader.py", line 299, in read_file
      return self.read_cpp_source_file(source_file)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/source_reader.py", line 318, in read_cpp_source_file
      xml_file = self.create_xml_file(ffname)
    File "/home/felipe/.local/lib/python3.6/site-packages/pygccxml/parser/source_reader.py", line 267, in create_xml_file
      ": %s status:%s" % (msg, exit_status))

  RuntimeError: Error occurred while running CASTXML: status:1

Call Stack (most recent call first):
  python/CMakeLists.txt:33 (generate_python_binding_source_code)

-- Configuring done
CMake Error at python/CMakeLists.txt:41 (add_library):
  Cannot find source file:

    /home/felipe/map/build/ad_map_access/python/AdMapAccessPythonWrapperPYTHON3.cpp

  Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp
  .hxx .in .txx

-- Generating done
-- Build files have been written to: /home/felipe/map/build/ad_map_access
berndgassmann commented 4 years ago

With the full output I see the compiler you use is gcc 8.4. I installed that one and tried on my machine. Also works.

But it seems to be a problem with two paths to the includes when I now see the output. /home/felipe/map/install/include/ad/map/point/GeoOperation.hpp /home/felipe/map/ad_map_access/python/../impl/include/ad/map/point/GeoOperation.hpp

So I guess you don't use colcon to build. But rather have run the plain example (https://github.com/carla-simulator/map/blob/master/doc/BUILDING.md#plain-cmake-build) I run the build with:

colcon build --metas colcon_python.meta

Then the ad_map_access include files get installed under: /home/felipe/map/install/ad_map_access/include/ad/map/point/GeoOperation.hpp

And because of that are NOT available to the python generation build (there only the ad_physics and ad_map_opendrive_reader and spdlog headers are visible).

But you seem to use one single common include folder where all is installed to and like this the installed version and the version to be processed are visible via two different paths. I believe strongly that this is the root cause; I have reproduced it that way!

So what you can do is: a) use colcon to build b) delete your installation folder again; and enable the python binding in the first cmake call on ad_map_access BEFORE the installation took place. Iif the install of ad_map_access already run through and you try to regenerate ... this leads to your failure c) when manually calling the build you make it as colcon does and install every package in its own subdir

I'll update the plain build docu to the option c) to get rid of this one.

felipecode commented 4 years ago

Thanks a lot @berndgassmann for your help ! The changes in c) fixed this problem and the python binding is successfully generated.

However, I am still not able to import the map access. It gives me an weird exception. Maybe i am doing some wrong with that.

I am also not able to run the tests.


>>> import libad_map_access_python3
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<std::vector<ad::map::point::ParaPoint, std::allocator<ad::map::point::ParaPoint> >, std::allocator<std::vector<ad::map::point::ParaPoint, std::allocator<ad::map::point::ParaPoint> > > > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<std::vector<ad::map::match::MapMatchedPosition, std::allocator<ad::map::match::MapMatchedPosition> >, std::allocator<std::vector<ad::map::match::MapMatchedPosition, std::allocator<ad::map::match::MapMatchedPosition> > > > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<std::shared_ptr<ad::map::intersection::Intersection>, std::allocator<std::shared_ptr<ad::map::intersection::Intersection> > > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::route::RoadSegment, std::allocator<ad::map::route::RoadSegment> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::route::LaneSegment, std::allocator<ad::map::route::LaneSegment> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::route::FullRoute, std::allocator<ad::map::route::FullRoute> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::restriction::SpeedLimit, std::allocator<ad::map::restriction::SpeedLimit> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::restriction::RoadUserType, std::allocator<ad::map::restriction::RoadUserType> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::restriction::Restriction, std::allocator<ad::map::restriction::Restriction> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::point::ParaPoint, std::allocator<ad::map::point::ParaPoint> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::point::GeoPoint, std::allocator<ad::map::point::GeoPoint> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::point::ENUPoint, std::allocator<ad::map::point::ENUPoint> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::point::ECEFPoint, std::allocator<ad::map::point::ECEFPoint> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::match::MapMatchedPosition, std::allocator<ad::map::match::MapMatchedPosition> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::match::LaneOccupiedRegion, std::allocator<ad::map::match::LaneOccupiedRegion> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::match::ENUObjectPosition, std::allocator<ad::map::match::ENUObjectPosition> > already registered; second conversion method ignored.
  return f(*args, **kwds)I tried using ctypes also to import the so 
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::lane::LaneId, std::allocator<ad::map::lane::LaneId> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::lane::ENUBorder, std::allocator<ad::map::lane::ENUBorder> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::lane::ContactType, std::allocator<ad::map::lane::ContactType> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::lane::ContactLocation, std::allocator<ad::map::lane::ContactLocation> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::lane::ContactLane, std::allocator<ad::map::lane::ContactLane> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::landmark::LandmarkId, std::allocator<ad::map::landmark::LandmarkId> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::landmark::ENULandmark, std::allocator<ad::map::landmark::ENULandmark> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::config::PointOfInterest, std::allocator<ad::map::config::PointOfInterest> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::vector<ad::map::access::PartitionId, std::allocator<ad::map::access::PartitionId> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::set<ad::map::lane::LaneId, std::less<ad::map::lane::LaneId>, std::allocator<ad::map::lane::LaneId> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::set<ad::map::landmark::LandmarkId, std::less<ad::map::landmark::LandmarkId>, std::allocator<ad::map::landmark::LandmarkId> > already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::route::ConnectingRouteType already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::ContactLocation already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::ContactType already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::intersection::IntersectionType already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::landmark::LandmarkType already registered; second conversion method ignored.getLanes
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::route::LaneChangeDirection already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::LaneDirection already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::LaneType already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::match::MapMatchedPositionType already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::match::ObjectReferencePoints already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::restriction::RoadUserType already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::route::RouteCreationMode already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::route::RouteSectionCreationMode already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::route::planning::RoutingDirection already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::route::ShortenRouteMode already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::route::ShortenRouteResult already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::landmark::TrafficLightType already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::landmark::TrafficSignType already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::access::TrafficType already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::intersection::TurnDirection already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::access::MapMetaData already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::access::PartitionId already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::config::MapEntry already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::config::PointOfInterest already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for std::shared_ptr<ad::map::intersection::Intersection> already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::landmark::ENULandmark already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::landmark::Landmark already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::landmark::LandmarkId already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::ContactLane already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::ECEFBorder already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::ENUBorder already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::GeoBorder already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::Lane already registered; second conversion method ignored.
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for ad::map::lane::LaneId already registered; second conversion method ignored.
  return f(*args, **kwds)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: initialization of libad_map_access_python3 raised unreported exception
>>> import libad_map_access_python3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: initialization of libad_map_access_python3 raised unreported exception

I tried using ctypes also to import the so library and it didnt work.

berndgassmann commented 4 years ago

Hi,

I tried to get to the bottom of this. The RuntimeWarning is triggered only if you (try) import the library twice, so the first import works somehow ... at least parts of it. When I don't import libad_physics_python3 beforehand, I also get the "raised unreported exception". But if you follow the documentation (https://github.com/carla-simulator/map/blob/master/doc/ad_map_access/ad_map_access_python.md) and first import ad_physics all should work fine:

import libad_physics_python3 as physics import libad_map_access_python3 as admap

To make it work in CARLA context, I call the init functions of all the libraries required explicitly in the module initialization https://github.com/carla-simulator/carla/blob/master/PythonAPI/carla/source/libcarla/libcarla.cpp#L227 which leads to https://github.com/carla-simulator/carla/blob/master/PythonAPI/carla/source/libcarla/AdRss.cpp#L112

I would say that this is rather a hack, but I don't know how one specifies in python dependencies correctly so that an import of libad_map_access_python3 would automatically import libad_physics_python3.

One might be able to apply the "hack" I did in carla also within this library directly to patch the generated code and enforce the call of the respective initlibad_physics_python2/PyInit_libad_physics_python3 from within; but long term it would perhaps make more sense to create some egg/wheel and upload that to some repo that you can install it via pip ....

felipecode commented 4 years ago

Thanks a lot for your help. This solved. I am sorry, i completely missed this part of the documentation.

berndgassmann commented 4 years ago

Hi, you don't have to say sorry. Your steps just make obvious what problems a user can face, so all improvements to usability are welcome. Rather I have to say sorry, that you were the one having to go through that pain now...

Bernd.