openstreetmap / tirex

Tirex tile queue manager. A drop-in replacement for renderd.
GNU General Public License v2.0
67 stars 23 forks source link

Needs better logging and other stuff #9

Open ajhalls opened 6 years ago

ajhalls commented 6 years ago

I have been trying to install this all day and there simply aren't enough log entries generated to identify problems. If I stop the service I can run it in debug mode, but not as root, so now I have 3 terminal windows open to start / stop the master and backend, and one for root so I can search for missing files in the debug output.

While the "Quick Start" has you use the command: tirex-batch --prio=1 map=test z=0 x=0 y=0, replacing the map name with example doesn't create the files, and doesn't create any errors at all. After working for a few hours tweaking permissions, I found a post on StackOverflow that told me you had to change the example.conf from: plugindir=/usr/lib/mapnik/input

to this: plugindir=/usr/lib/mapnik/3.0/input

There was no logs anywhere to indicate missing / misconfigured settings.

After working all day, I got it rendering about 20% of the example map at first, and if I keep scrolling around I can get more, but there isn't anything that I can see that indicates why the other sections are missing. capture

While I do see files filling up the /var/lib/tirex/tiles/example directory, when I refresh the map, it isn't pulling up more things than it did last time, it doesn't seem to be caching it at all.

Determined to make this work, I persist assuming I could pre render all the tiles using tirex-batch, but as I look at how to expand the example.xml, there doesn't seem to be any documentation or a more complete example.

I looked at the renderd mapnik.xml which has quite a bit of stuff in it, but not sure how much of that transfers over to Tirex. When I tried just copying it all over, I got tons of errors about missing fonts and such.

While the description describes Tirex as being more flexible and so on, it is hard to see it being ready for anyone outside of the development team to use it as so much is left undocumented. Where do I get shape files, do I need shape files, how do I connect it to the postgresql data? How do I define more complex layers and styles than a simple coastline map?

I do appreciate your work, and it does seem like maybe it would be faster than renderd, but it is hard to tell since there isn't a better xml example.

woodpeck commented 6 years ago

Have you checked the syslog output (/var/log/syslog, /var/log/messages, depending on your system configuration) at all? This is where you should see messages like "cannot load style X because no plugins have been loaded for shape file processing" or so. If you have not checked the syslog, then I guess the docs need to be improved to tell people to check the syslog. If you have checked the syslog and found nothing, then maybe there's a bug in the logging.

As for caching or not caching things, this is outside of the scope of Tirex, and handled by mod_tile which you likely have also installed - mod_tile will check if a tile you request is in the cache, and only instruct Tirex to render afresh if it is not, or too old. mod_tile logs to the Apache log. Also, running tirex-status while you browse the map will give you an idea of what's happening.

As for writing nicer map styles, needing shape files or not, etc. this is also out of Tirex's scope; the Tirex mapnik rendering backend will consume any Mapnik xml that you throw at it, and whether you need shapes or a database and whether your sea is green or blue, is all controlled by that Mapnik style file. Even though you can write it by hand in a text editor, nowadays Mapnik style files are often designed using the "Carto CSS" language, and converted to Mapnik XML once you're happy with your style. kosmtik or Mapbox Studio Classic are tools frequently used in map styling.

If you were unaware of these distinctions, and thought that Tirex was essentially the one-stop shop for nice maps rather than just a component of a larger tool chain, perhaps you could suggest where in the docs we should place a huge red message that explains things better, and I'll endeavour to improve that.

ajhalls commented 6 years ago

I did check the syslog and enabled /var/log/messages. It was very odd that it would render some, but not all. It was teasing me almost like it might work, but then not.

I was able to get the mapbox.xml working by moving the shape files, symbols and fonts to /usr/share/tirex/example-map/.

I understand that it is outside the scope to do everything, but a simple comment in the Quick Start to copy those files over would have helped. Having a clearer picture of where tirex fits into the toolchain would also help. There are so many moving parts, it is hard to know where one starts and the other begins when it all fails.

In the end, I went back to renderd because it worked. It might be slower sometimes in your tests, but once it renders something, it works the next time I reload the page. For me renderd was faster in almost all cases.

Kosmtik did have the distinct honor of using the most memory and CPU and starting the most processes on my 16 core 3.3GHz 192GB RAM server, but it didn't seem fast at all. Sometimes I would zoom in to Florida zoom level 12 and wait a good minute for it to load. If I am just using it to create an xml file, then speed isn't really an issue, but it was interesting.