Error with create_linestring

[lucadelu]

Hi all,

I have a problem creating linestring from ways, my code was working until few days ago and now return error for each way during the create_linestring call, the error is "need at least two points for linestring" but I don't understand why since my osm file seems to be correct.

The code is the following

class CaiRoutesHandler(osmium.SimpleHandler):
    """Class to parse CAI routes from OSM file and return them in different
    format"""
    def __init__(self, separator=","):
        super(CaiRoutesHandler, self).__init__()
        self.ways = {}
        self.sep = separator

    def way(self, way):
        """Function to parse ways"""
        print(way)
        self.ways[way.id] = {}
        try:
            self.ways[way.id]['geom'] = WKTFAB.create_linestring(way)
        except Exception:
            print("Error creating geometry for way {}".format(way.id))
            pass
        self.ways[way.id]['tags'] = way.tags

Any idea about what could be wrong? How can I debug it?

Thanks Luca

[joto]

Are you sure you have all the nodes needed for the ways?

[lucadelu]

I tryed some of the failing ways and they have all nodes.

I downloaded the data using overpass api with the following query

[timeout:2500][out:xml];
area["name"="La Spezia"]->.a;

(relation
  ["route"="hiking"]

  ["cai_scale"]
  (area.a);
);

out body;
>;
out body qt;

you could try by your self if you want to try to check if it is working for you.

[wiktorn]

Overpass doesn't guarantee that all the nodes comes before the ways. Especially in your query, nodes will come last. You need to sort your input first, e.g. running through osmium.MergeInputReader, write to temporary file, and then again, parse the file with your handler.

You can see an example here

[lucadelu]

@wiktorn thanks for suggestion. I'm trying to use osmium.MergeInputReader but I get strange behavior , the code is stopped at mir.add_buffer (this function is printing the xml data in stdout too). Do you have any idea about this problem?

mir = osmium.MergeInputReader()
mir.add_buffer(data, "osm")
# the code stop here after printing the xml osm data
with tempfile.NamedTemporaryFile(suffix=".osm") as temp_osm:
    os.unlink(temp_osm.name)
    wh = osmium.WriteHandler(temp_osm.name)
    mir.apply(wh)
    wh.close()
    output = temp_osm.read()

[lonvia]

Have you enabled location parsing when you execute the handler?

[lucadelu]

@lonvia no I didn't enable it, where should I have to enable location? in the MergeInputReader or in mine CaiRoutesHandler class?

I did some tests and I didn't find the right place...

[lonvia]

Where ever you call apply(), see https://docs.osmcode.org/pyosmium/latest/intro.html#handling-geometries .

[lonvia]

So @wiktorn example seems a bit overcomplicated. If you have data from Overpass do something along those lines:

cai_handler = CaiRoutesHandler()

data = get_xml_file_from_overpass()

mir = osmium.MergeInputReader()
mir.add_buffer(data, "osm")

mir.apply(cai_handler, idx="flex_mem", simplify=True)

Adding the idx parameter ensures that the locations are cached and processed.

[lucadelu]

@lonvia location in apply_file in CaiRoutesHandler seems to fix my problem. However is it better to use your piece of code or I can trust in location=yes and overpass-api data output?

[lonvia]

No you can't. Pyosmium's location cache only works correctly when the data is sorted. Apparently that is not the case for overpass data.

[joto]

Maybe it is easiest to use osmium sort on the Overpass output first, then you don't have to deal with sorting in your Python program.

[wiktorn]

@lonvia my solution might be a bit overcomplicated for this particular case (linestring from ways), but it's handling general case properly.

Your solution doesn't call area handler where mine does if input set contains multipolygons. Even for sorted input data as far as I run my tests.

@lucadelu Check what kind of object do you pass. You need to pass byte/bytearray like object, not string into add_buffer, otherwise pybind11 prints error message that for your input it didn't found matching method and it prints out the values of arguments. That's why you may see your input printed.

[lucadelu]

So @wiktorn example seems a bit overcomplicated. If you have data from Overpass do something along those lines:

cai_handler = CaiRoutesHandler()

data = get_xml_file_from_overpass()

mir = osmium.MergeInputReader()
mir.add_buffer(data, "osm")

mir.apply(cai_handler, idx="flex_mem", simplify=True)

Adding the idx parameter ensures that the locations are cached and processed.

Fixed using this code!