search

mapnik / python-mapnik

Python bindings for mapnik
GNU Lesser General Public License v2.1
160 stars 91 forks source link

Debian packaging #30

Open springmeyer opened 9 years ago

springmeyer commented 9 years ago

@kapouer - Hello. I see you are starting to work on packaging the upcoming Mapnik 3.0.0 release in debian. That is awesome, thank you.

In terms of the python bindings, they should be a separate package now. Let's use this ticket to discuss how we can help with debian packaging.

kapouer commented 9 years ago

Hello, for now i have seen these two problems. Anything else was already taken care of with mapnik 2.2 packaging.

sebastic commented 9 years ago

I just sent an update to the thread @kapouer linked above:

Initial packaging for python-mapnik is available on Alioth:

http://anonscm.debian.org/cgit/pkg-grass/python-mapnik.git

Unfortunately the code in the python-mapnik git repository is not compatible with mapnik 3.0.0-RC3.

It currently fails to build with:

src/mapnik_font_engine.cpp:37:37: fatal error: mapnik/util/singleton.hpp: No such file or directory

This header is no longer included in mapnik 3.

flippmoke commented 9 years ago

@sebastic @kapouer We probably should release another RC, but there was some code clean up that @artemp and @springmeyer have performed since RC3 and the current python-mapnik only works off master. @springmeyer is reviewing https://github.com/mapnik/mapnik/pull/2904, once that is merged we can either expect an official Mapnik 3.0.0 or another RC.

springmeyer commented 9 years ago

Okay v3.0.0 tag is out: http://mapnik.org/news/release-3.0.0/

@kapouer and @sebastic - let us know if you have any issues working with the v3.0.0 tag. We'll be prepared to issue a v3.0.1 or v3.1.0 if you hit anything needing fixed (so you don't need to carry patches).

i plan on not distributing mapnik 3 on these architectures

@kapouer - that is absolutely fine.

sebastic commented 9 years ago

The build looks much better with Mapnik 3.0.0, but it still fails.

It looks like it requires custom boost libraries (because of the unusual -mt suffix)?

c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-2.7/src/mapnik_color.o build/temp.linux-x86_64-2.7/src/mapnik_coord.o build/temp.linux-x86_64-2.7/src/mapnik_datasource.o build/temp.linux-x86_64-2.7/src/mapnik_datasource_cache.o build/temp.linux-x86_64-2.7/src/mapnik_envelope.o build/temp.linux-x86_64-2.7/src/mapnik_expression.o build/temp.linux-x86_64-2.7/src/mapnik_feature.o build/temp.linux-x86_64-2.7/src/mapnik_featureset.o build/temp.linux-x86_64-2.7/src/mapnik_font_engine.o build/temp.linux-x86_64-2.7/src/mapnik_fontset.o build/temp.linux-x86_64-2.7/src/mapnik_gamma_method.o build/temp.linux-x86_64-2.7/src/mapnik_geometry.o build/temp.linux-x86_64-2.7/src/mapnik_grid.o build/temp.linux-x86_64-2.7/src/mapnik_grid_view.o build/temp.linux-x86_64-2.7/src/mapnik_image.o build/temp.linux-x86_64-2.7/src/mapnik_image_view.o build/temp.linux-x86_64-2.7/src/mapnik_label_collision_detector.o build/temp.linux-x86_64-2.7/src/mapnik_layer.o build/temp.linux-x86_64-2.7/src/mapnik_logger.o build/temp.linux-x86_64-2.7/src/mapnik_map.o build/temp.linux-x86_64-2.7/src/mapnik_palette.o build/temp.linux-x86_64-2.7/src/mapnik_parameters.o build/temp.linux-x86_64-2.7/src/mapnik_proj_transform.o build/temp.linux-x86_64-2.7/src/mapnik_projection.o build/temp.linux-x86_64-2.7/src/mapnik_python.o build/temp.linux-x86_64-2.7/src/mapnik_query.o build/temp.linux-x86_64-2.7/src/mapnik_raster_colorizer.o build/temp.linux-x86_64-2.7/src/mapnik_rule.o build/temp.linux-x86_64-2.7/src/mapnik_scaling_method.o build/temp.linux-x86_64-2.7/src/mapnik_style.o build/temp.linux-x86_64-2.7/src/mapnik_svg_generator_grammar.o build/temp.linux-x86_64-2.7/src/mapnik_symbolizer.o build/temp.linux-x86_64-2.7/src/mapnik_text_placement.o build/temp.linux-x86_64-2.7/src/mapnik_view_transform.o build/temp.linux-x86_64-2.7/src/python_grid_utils.o -lmapnik -lmapnik-wkt -lmapnik-json -lboost_python-mt -lboost_thread-mt -lboost_system-mt -o /tmp/buildd/python-mapnik-0.0~20150708-c005502/.pybuild/pythonX.Y_2.7/build/mapnik/_mapnik.so -L/usr/lib -lmapnik -L/usr/lib -L/usr/lib/x86_64-linux-gnu -Wl,-z,relro -g0 -pthread -lrt -Wl,-z,origin -Wl,-rpath=$ORIGIN
/usr/bin/ld: cannot find -lmapnik-wkt
/usr/bin/ld: cannot find -lmapnik-json
/usr/bin/ld: cannot find -lboost_python-mt
/usr/bin/ld: cannot find -lboost_thread-mt
/usr/bin/ld: cannot find -lboost_system-mt
collect2: error: ld returned 1 exit status
error: command 'c++' failed with exit status 1
springmeyer commented 9 years ago

@sebastic - sorry about that breakage. In terms of the boost linking problems this is https://github.com/mapnik/python-mapnik/issues/33. As far as the linking problem with -lmapnik-wkt and -lmapnik-json that is a new problem I've not seen. Those are static libs built by Mapnik core. So the problem is likely that the Mapnik 3.0.0 core package needs to install them.

sebastic commented 9 years ago

Thanks for the hint about the static libs, I've updated the mapnik package to include those in the -dev package too.

@kapouer, can you have a look at those changes?

kapouer commented 9 years ago

@sebastic seems all right (unrelated: a manpage for nik2img ?) @springmeyer python-mapnik need access to mapnik-wkt, mapnik-json internals, but these two libs are actually in the mapnik shared lib ? Or is there another reason to having separate static libs ?

springmeyer commented 9 years ago

@sebastic seems all right (unrelated: a manpage for nik2img ?)

Can I ask that you don't package nik2img yet? For 3.1.0 I plan to create a new command line tool called mapnik that will have a render command. So the oddly named nik2img map.xml image.png will become mapnik render map.xml image.png. After that would be great to package.

@springmeyer python-mapnik need access to mapnik-wkt, mapnik-json internals, but these two libs are actually in the mapnik shared lib?

They are separate and not part of libmapnik. The reason is that they would add a couple extra MB to libmapnik and are not needed by any code in libmapnik. So they are external libs to allow apps that need JSON and WKT i/o to include them. The python bindings do depend on them.

kapouer commented 9 years ago

@springmeyer https://github.com/springmeyer python-mapnik need access to mapnik-wkt, mapnik-json internals, but these two libs are actually in the mapnik shared lib?

They are separate and not part of libmapnik. The reason is that they would add a couple extra MB to libmapnik and are not needed by any code in libmapnik. So they are external libs to allow apps that need JSON and WKT i/o to include them. The python bindings do depend on them.

@springmeyer so it'd make sense to build them as shared libs as well, no ?

springmeyer commented 9 years ago

@kapouer they are intentionally only static libs so that applications that link them will be able to discard any functions not used. So I would prefer not providing them as shared libraries. But yes theoretically they could be built as shared libs as well. A little more background is that this json/wkt code would normally be header-only. So just .hpp files: that allows for any apps being compiled against them to get just what they need without the complexity of another library to link against. But in this case the code for json/wkt parsing and output uses boost spirit - notorious for slow and memory intensive compile times. So, to save time recompiling they can be (in addition to included as header-only) compiled as libraries. I've done that to help speed up the node-mapnik/python-mapnik compile since I do this compile pretty frequently.

To recap: I understand and respect the debian shared library protocol. But making shared libraries would just add complexity I would prefer not to support. Let me know what you think.

kapouer commented 9 years ago

@kapouer https://github.com/kapouer they are intentionally only static libs so that applications that link them will be able to discard any functions not used. So I would prefer not providing them as shared libraries. But yes theoretically they could be built as shared libs as well. A little more background is that this json/wkt code would normally be header-only. So just .hpp files: that allows for any apps being compiled against them to get just what they need without the complexity of another library to link against. But in this case the code for json/wkt parsing and output uses boost spirit - notorious for slow and memory intensive compile times. So, to save time recompiling they can be (in addition to included as header-only) compiled as libraries. I've done that to help speed up the node-mapnik/python-mapnik compile since I do this compile pretty frequently.

To recap: I understand and respect the debian shared library protocol. But making shared libraries would just add complexity I would prefer not to support. Let me know what you think.

@springmeyer thank you for the explanation (which i'm using in the README.Debian); actually debian policy allows static libs in that kind of scenario.

sebastic commented 9 years ago

With the recent changes for Python 3 (#36) the python-mapnik Debian package could drop most of its patches. The remaining patch is for the Boost library names (#12).

The only thing we're really missing for the Debian package are proper upstream tarball releases (tagged in git and/or published on pypi). I'm looking forward to the Release 1.0 milestone for that.

This issue is mostly resolved with the python-mapnik packaging as mentioned before. Some related links:

yohanboniface commented 9 years ago

The remaining patch is for the Boost library names (#12).

@sebastic about #12, have you had a chance to test #48?

sebastic commented 9 years ago

@yohanboniface I hadn't yet, but I've now replaced my simple patch with yours from #48 and it seems to work very well for both Python 2.7 and 3.4. It will be included in the next upload.

yohanboniface commented 9 years ago

Great, good to know, thanks :)