systemed / tilemaker

Make OpenStreetMap vector tiles without the stack
https://tilemaker.org/
Other
1.42k stars 228 forks source link

Latest version does not work on windows #661

Closed Wicpar closed 6 months ago

Wicpar commented 6 months ago

it just stops after one secons. 2.4.0 seems to work

PS \tilemaker-windows\build\RelWithDebInfo> ./tilemaker.exe --verbose --store ./store --config .\config-openmaptiles.json --process .\process-openmaptiles.lua --input planet-240115.osm.pbf --output tiles
Layer place (z0-14)
Layer boundary (z0-14)
Layer poi (z12-14)
Layer poi_detail (z14-14) -> poi
Layer housenumber (z14-14)
Layer waterway (z8-14)
Layer waterway_detail (z12-14) -> waterway
Layer transportation (z4-14)
Layer transportation_name (z8-14)
Layer transportation_name_mid (z12-14) -> transportation_name
Layer transportation_name_detail (z14-14) -> transportation_name
Layer building (z13-14)
Layer water (z6-14)
Layer ocean (z0-14) -> water
Layer water_name (z14-14)
Layer water_name_detail (z14-14) -> water_name
Layer aeroway (z11-14)
Layer aerodrome_label (z10-14)
Layer park (z11-14)
Layer landuse (z4-14)
Layer urban_areas (z4-8) -> landuse
Layer landcover (z0-14)
Layer ice_shelf (z0-9) -> landcover
Layer glacier (z2-9) -> landcover
Layer mountain_peak (z11-14)
Bounding box -180, -85.06, 180, 85.06
PS \tilemaker-windows\build\RelWithDebInfo>
Star-42 commented 6 months ago

Same here. tilemaker just terminates after logging configured layers.

systemed commented 6 months ago

I have just installed Windows on my Mac laptop and will see what I can work out. It won’t be quick as I’ll have to get VS Code, MSVC++ etc. up and running even before the debugging. But it’s on the radar.

qmjy commented 6 months ago

same problem on v3.0.0

Star-42 commented 6 months ago

@systemd: Thanks for your quick response! Meanwhile I manged to fix the issue.

source file: src/pbf_processor.cpp method: PbfHasOptionalFeature bool PbfHasOptionalFeature(const std::string& inputFile, const std::string& feature) { std::ifstream infile(inputFile, std::ifstream::in); auto header = reader.readHeaderFromFile(infile); infile.close(); return header.optionalFeatures.find(feature) != header.optionalFeatures.end(); }

change: replace std::ifstream infile(inputFile, std::ifstream::in); with fstream infile(inputFile, ios::in | ios::binary);

systemed commented 6 months ago

Excellent, thanks. I'll have a play around adding that and put a patch together in the next few days.

freeExec commented 6 months ago

Were there any problems with:

The application crashes because accesses an index outside the array limit.

https://github.com/systemed/tilemaker/blob/6e335efdd13d876f4ddacd0368ee04811337c8ac/src/sorted_node_store.cpp#L380 https://github.com/systemed/tilemaker/blob/6e335efdd13d876f4ddacd0368ee04811337c8ac/src/sorted_node_store.cpp#L426

systemed commented 6 months ago

Were there any problems with:

The application crashes because accesses an index outside the array limit.

I haven't seen this - what are you encountering it with? (OS, command line, source data etc.)

freeExec commented 6 months ago

Windows (MSVC)

On any given data, it accesses elements equal to the size of the array. Another thing is that in the release build, if there is an error, the thread is silently terminated. The error becomes clear if you do a debug build.

cldellow commented 6 months ago

I agree that's undefined behaviour due to an out of bounds read. The loop is dangerous; it uses <= on purpose for other reasons, but it's a footgun, as seen here.

I think the correct fix is to skip the last 3 lines in the loop when i == nodes.size(), as no one actually consumes them. I'm running some experiments now to confirm and will send a PR if it looks good.