Python 3 status

[yohanboniface]

I can't find any mention of the supported python versions. Is python 3 (=3.4) supported yet? If not, is it on the pipe?

Thanks for your lights :)

[springmeyer]

Yes it is supported. However it has not see much testing. Next step would be to start testing it in travis.

[yohanboniface]

Excellent news!

However it has not see much testing. Next step would be to start testing it in travis.

I guess you mean python unittests (no C++ ones) ? If yes I may give a hand.

[flippmoke]

Yes, this would be the python unit tests, not the C++ ones :+1:

[yohanboniface]

ok :) Is it "just" about making Travis run them both in python 2.7 and python 3.4 (and maybe others), or is there a specific area that needs more unittests to be well covered?

[flippmoke]

@yohanboniface both?

We need to integrate coveralls into our testing so that we can see where we need to write more tests in the python bindings, but we also just need to test in general more with python 3.x by integrating a test of it in the travis script.

https://github.com/mapnik/python-mapnik/issues/37

[sebastic]

I've enabled Python 3 support in the Debian packaging (#30), but it fails to build:

I: pybuild base:170: python3.4 setup.py config 
Traceback (most recent call last):
  File "setup.py", line 38, in <module>
    linkflags = subprocess.check_output([mapnik_config, '--libs']).rstrip('\n').split(' ')
TypeError: 'str' does not support the buffer interface

These errors are relatively easy to fix with changes like these:

linkflags = subprocess.check_output([mapnik_config, '--libs']).rstrip(b'\n').split(b' ')

But that just moves the problem along:

I: pybuild base:170: python3.4 setup.py config
Traceback (most recent call last):
  File "setup.py", line 156, in <module>
    os.environ["CC"] = subprocess.check_output([mapnik_config, '--cxx']).rstrip(b'\n')
  File "/usr/lib/python3.4/os.py", line 638, in __setitem__
    value = self.encodevalue(value)
  File "/usr/lib/python3.4/os.py", line 706, in encode
    raise TypeError("str expected, not %s" % type(value).__name__)
TypeError: str expected, not bytes

That can be fixed with:

os.environ["CC"] = subprocess.check_output([mapnik_config, '--cxx']).rstrip(b'\n').decode("utf-8")

And then the build with Python 2.7 fails:

Traceback (most recent call last):
  File "setup.py", line 225, in <module>
    extra_link_args = linkflags,
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/distutils/command/build.py", line 128, in run
    self.run_command(cmd_name)
  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/build_ext.py", line 50, in run
    _build_ext.run(self)
  File "/usr/lib/python2.7/distutils/command/build_ext.py", line 337, in run
    self.build_extensions()
  File "/usr/lib/python2.7/distutils/command/build_ext.py", line 446, in build_extensions
    self.build_extension(ext)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/build_ext.py", line 183, in build_extension
    _build_ext.build_extension(self, ext)
  File "/usr/lib/python2.7/distutils/command/build_ext.py", line 496, in build_extension
    depends=ext.depends)
  File "/usr/lib/python2.7/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/usr/lib/python2.7/distutils/unixccompiler.py", line 119, in _compile
    self.spawn(compiler_so + cc_args + [src, '-o', obj] +
TypeError: coercing to Unicode: need string or buffer, list found

Here I gave up trying to patch to build.

I'm looking forward to improved Python 3 support so I can re-enable it in the Debian package.

[yohanboniface]

I've a patch for this. I was trying to finish to install and run tests to be sure the patch is complete, but I've hit some system dependencies issues, and then switched to another task for today. But I'm sure the patch fixes the p3 syntax of setup.py. Should I submit it yet? Or could we wait some days so I have time to finish my install and run tests before?

[yohanboniface]

Pushed the patch on a branch, so you can use it if needed :)

[songololo]

Building mapnik (as opposed to python-mapnik) from source on a Mac I encountered the issue that scons is required, which is not compatible with Python 3. I'm curious how others have built to Python 3? Thanks.

[yohanboniface]

Building from source on a Mac

How have you built python-mapnik?

[artemp]

@shongololo - have you tried

python3 setup.py develop

? The above works for me on OS X 10.11 + clang + Python 3.5

NOTE: You need to build boost-python with Python3 support

[songololo]

@artemp @yohanboniface Tried downloading from python-mapnik but when I run setup it tells me it can't find the linking flags from mapnik config:

Traceback (most recent call last):
  File "setup.py", line 113, in <module>
    linkflags = check_output([mapnik_config, '--libs']).split(' ')
  File "setup.py", line 19, in check_output
    output = subprocess.check_output(args)
  File "/Users/shongololo/anaconda/lib/python3.5/subprocess.py", line 629, in check_output
    **kwargs).stdout
  File "/Users/shongololo/anaconda/lib/python3.5/subprocess.py", line 696, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/Users/shongololo/anaconda/lib/python3.5/subprocess.py", line 950, in __init__
    restore_signals, start_new_session)
  File "/Users/shongololo/anaconda/lib/python3.5/subprocess.py", line 1540, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'mapnik-config'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 117, in <module>
    raise Exception("Failed to find proper linking flags from mapnik config")
Exception: Failed to find proper linking flags from mapnik config

[tomhughes]

I think you're talking at cross purposes here - @shongololo wants to know how to build mapnik itself without python 2 (given that it uses scons, which he is saying doesn't work with python 3) not whether the python bindings here support python 3 (they do).

[songololo]

@tomhughes thanks for clarifying. I've edited my original question to clarify that it is mapnik itself that I'm trying to build.

[songololo]

Bumping this thread, hoping for some info on how to build Mapnik itself with Python 3?

[artemp]

@shongololo - once http://scons.org/ adds support for Python3 we can start testing/updating our build scripts.

[songololo]

@artemp am I of the correct understanding that mapnik built via scons and Python 2.7 will work from python-mapnik run through Python 3?

[artemp]

@shongololo - yes

[alexandervlpl]

Whether using the python3-mapnik debian package or compiling from source with python3, the Map object has none of the functions I need: zoom_max(), set_center_and_zoom(). Where did they all go? Does anyone actually know how to use the python3 version?

[artemp]

Whether using the python3-mapnik debian package or compiling from source with python3, the Map object has none of the functions I need: zoom_max(), set_center_and_zoom(). Where did they all go? Does anyone actually know how to use the python3 version?

I don't think they are from python-mapnik ^^

Python 3.9.7 (default, Sep 10 2021, 14:59:43) 
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mapnik
>>> map=mapnik.Map(200,200)
>>> dir(map)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__instance_size__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'append_fontset', 'append_style', 'aspect_fix_mode', 'background', 'background_color', 'background_image', 'background_image_comp_op', 'background_image_opacity', 'base', 'buffer_size', 'buffered_envelope', 'envelope', 'find_fontset', 'find_style', 'height', 'layers', 'maximum_extent', 'pan', 'pan_and_zoom', 'parameters', 'query_map_point', 'query_point', 'remove_all', 'remove_style', 'resize', 'scale', 'scale_denominator', 'srs', 'styles', 'view_transform', 'width', 'zoom', 'zoom_all', 'zoom_to_box']

[alexandervlpl]

@artemp You're right. :man_facepalming: In case somebody else has this problem: I was using the old nik2img stuff and these methods actually come from its fancy metaclass injectors that fail silently in Python 3.