Bundles, descriptions and issues

[polarkernel]

The first task in the search for objects of the class Bundle is the search for equidistant path segments. For a better understanding of the concept and its problems, I will first explain how the algorithm works.

First, all way-sections that exceed a minimal length and are not too curved are entered into a static spatial index. This index uses bounding boxes around the sections to find neighbor sections. Way-sections, whose bounding boxes overlap, are candidates as neighbors. For every section (template) in this index, its neighbors are searched and tested, whether they are equidistant to the template section. This is done as follows:

The black line is the centerline of the template section. A buffering polygon is constructed around this line, spaced according to the category of the section (red dotted polygon). Assume that the cyan line is the centerline of the neighbor candidate. A clipping algorithm first determines the length inLineLength of the candidate within the buffer polygon. In this example, this is the length between p1 and p2. This length must at least be 10 % of the total length of the candidate.

However, a short candidate line could be perpendicular to the template, but almost completely within the buffer polygon. So we need an additional condition: The distances d1 and d2 between the entry and exit points p1 and p2 and the template are computed. Then, some kind of slope relative to the template is determined from the difference between these distances and the length inLineLength of the candidate within the buffer polygon to as slope = abs(d1-d2)/inLineLength. Demanding a small value for slope rules such cases out.

This algorithm has the advantage, that overlapping way-sections continue the equidistant parts, as long as only one side is interrupted. For instance these three way-sections (black lines with red dotted surrounding buffer polygon) would be considered to belong to the same group (Bundle):

However, there is a disadvantage, when the intersections are almost perpendicular, so that this effect does not work anymore. In the image below on the left (middle part in _rotterdam01.osm) this effect is clearly visible. Overlapping sections result in parallel ways through the intersection (all the same color). In the image on the right (intersection between Karl-Marx-Allee and Straße der Pariser Kommune in _berlin_karl_marxallee.osm), the ways are almost perpendicular, and the groups always contain only the short parallel parts (same color is same group).

[vvoovv]

However, a short candidate line could be perpendicular to the template, but almost completely within the buffer polygon.

I don't understand this case. If the short candidate is perpendicular to the template and completely or always completely within the buffer polygon, then there is no intersection or only one intersection of the short candidate with the buffer polygon. So both d1 and d2 can not be calculated or only either of d1 or d2 can be calculated.

[vvoovv]

A buffering polygon is constructed around this line.

On a separate note. To construct the buffering polygon, the original curve must be offsetted. Do you check for self-intersections when offsetting the curve? Do you process the self-intersections? I need this feature to render roadways and sidewalks correctly.

[polarkernel]

I don't understand this case.

The case with the intersection points p1 and p2, which I have drawn above, is only a general illustration. Often, the equidistant candidates are completely inside the buffer polygon, like candidateA in the figure below. Then, the endpoints of the candidate line are used as the entry and exit points of the "inside" part. Because the difference between the distances of these points to the template line is very small compared to the inside length of the candidate, the value of slope gets small.

In this example, you can also see why I called this value slope. If we straightened the template line to a straight horizontal line and drew a straight line using the distances d1 and d2 for the points p1 and p2 of the candidate line, the value of slope would be exactly the slope of the candidate line.

For the other case with candidateB, the difference of the distances from the entry and exit points on the candidate line to the template line is large compared to the inside length of the candidate, the value of slope becomes large. Such a line can't be accepted as parallel (equidistant).

[polarkernel]

Do you check for self-intersections when offsetting the curve? Do you process the self-intersections?

No, not completely. Self-intersections are only processed between adjacent line segments. For the equidistant line detection application, this is not important because lines that are too curved are already excluded from the static spatial index. In addition, I successfully replaced corners with a Corner class, as discussed at the end of https://github.com/prochitecture/blosm/issues/72#issuecomment-1874250570.

Some time ago, I made an attempt to use a different algorithm that gave better results, as far as I can remember. I will try to find it and let you know if I do.

[vvoovv]

I actually meant this kind of self-intersections:

[polarkernel]

I actually meant this kind of self-intersections:

I think these are computed correctly, as expected for an offset line:

[vvoovv]

I think these are computed correctly, as expected for an offset line

Could you please paste the code to generate the image above. I'd like to know the API calls to generate an offset for a curve with processing of self-intersections.

[polarkernel]

The code was:

from mathutils import Vector
import matplotlib.pyplot as plt
from lib.CompGeom.PolyLine import PolyLine

line = [
    Vector((1.8,3)),
    Vector((1,1.8)),
    Vector((0.2,1)),
    Vector((0,0)),
    Vector((0.2,-1)),
    Vector((0.5,-2)),
    Vector((0.8,-3))
]

polyline = PolyLine(line)
polyline.plot('k',2)

for d in range(20):
    dist = 0.2*(d+1)
    offsetPoly = polyline.buffer(dist,dist)

    x = [n[0] for n in offsetPoly] + [offsetPoly[0][0]]
    y = [n[1] for n in offsetPoly] + [offsetPoly[0][1]]
    plt.plot(x[:-1],y[:-1],'r',linewidth=0.5)
    plt.plot(x[:-1],y[:-1],'b.',markersize=2)

plt.gca().axis('equal')
plt.show()

[vvoovv]

I noticed that extra points are created for convex corners. That rounds the resulting corners. Is it possible to turn the rounding mode off?

[polarkernel]

I noticed that extra points are created for convex corners. That rounds the resulting corners. Is it possible to turn the rounding mode off?

Well, for now just a patch: In module lib/CompGeom/OffsetGenerator.py, replace line 91

fillet = self.constructFillet(p2,seg[1],p3,abs(self.distance))

by

fillet = None #self.constructFillet(p2,seg[1],p3,abs(self.distance))

Should you like to use this feature permanently, we will have to introduce some function argument.

[polarkernel]

Some time ago, I made an attempt to use a different algorithm that gave better results, as far as I can remember. I will try to find it and let you know if I do.

I found this old code. It implements a totally different algorithm. It has a function for lines and a function for polygons. As far as I remember, there were some minor floating-point accuracy issues (see red arrow), but it actually works on a much larger parameter range than the one that is in the addon. It is also capable of creating holes, as shown in the examples below, which is not the case with the current one. I am convinced that I would be able to finish it and make it robust and reliable if needed.

[vvoovv]

However, there is a disadvantage, when the intersections are almost perpendicular, so that this effect does not work anymore.

Did I understand the problem correctly that it's not possible to join the adjacent bundles for perpendicular way-sections since the buffering polygons do not intersect in that case?

Adjacent way-sections of the bundle can be checked if they are collinear with the way-sections of the bundle and that they also form a bundle.

[polarkernel]

Did I understand the problem correctly that it's not possible to join the adjacent bundles for perpendicular way-sections since the buffering polygons do not intersect in that case?

I should have written difference instead of disadvantage. For now, this is just an observation that may or may not play a role later. I will try to build a more complete set of observations within this thread. For the moment it does not make sense to build a theory, an intersection may also look like this (on the left of _osm_extracts/streets/milano01.osm):

[polarkernel]

I made a first attempt to find clustered intersections. It is quite simple and works as follows:

• Collect all the endpoints of sections that have been identified as equidistant into a set so that each vertex appears only once.
• Apply the DBSCAN algorithm to this set of endpoints. This algorithm groups closely-spaced points (points with neighbors within a given maximum distance) into point clusters, which can then be used as nuclei for clustered intersections.

The result looks already quite promising:

Looking at the images of the intersection between Karl-Marx-Allee and Straße der Pariser Kommune in _berlin_karl_marxallee.osm (left) and of the intersection in the middle part of _rotterdam01.osm (right), you can notice a problem, that will have to be discussed (the dotted polygon is the convex hull of all point detected as one group):

In both cases, all intersections belonging to this cluster have been detected, including those between pedestrian and bicycle ways. The question now is, where do the bundles (Bundle class) and the intersections (Intersection class) start? What about crosswalks? Do they belong to the bundles, and what about their intersections?

Filtering out all these details is a major task that now awaits us. Not all crossings are as classically shaped as these two examples above:

[vvoovv]

Not all crossings are as classically shaped

What if intersections with footways and cycleways are ignored? Will the intersections have a more classical shape?

[vvoovv]

What about crosswalks? Do they belong to the bundles, and what about their intersections?

Originally I thought that a crosswalk is a part of a Roadway (see also https://github.com/prochitecture/blosm/issues/72#issuecomment-1846141098). Filets on intersections make the things more complicated. The question can be also raised if a filet should be a part of an intersection.

The crosswalks are located on the filets on the intersection below:

I'll think what to do with the filets in that context.

[polarkernel]

What if intersections with footways and cycleways are ignored? Will the intersections have a more classical shape?

Yes. It should also be simple to detect them. I have more concerns about their assignment, as you also noted:

The question can be also raised if a filet should be a part of an intersection.

Locating fillets on bundles can be ambiguous. For example, the one at the top right might belong to the upward or to the right-hand bundle.

[polarkernel]

What if intersections with footways and cycleways are ignored? Will the intersections have a more classical shape?

I committed a version in which intersections between categories of minor roads such as "pedestrian", "track", "footpath", "path", "cycleway" or "bridleway" are distinguished from intersections with major roads. The areas of the two are drawn as convex hull in different saturations, and the intersections of the former are drawn as rectangles and the latter as circles:

[vvoovv]

I also suggest drawing a convex hull for "roadway" intersections (primary, secondary, tertiary, residential, etc).

[polarkernel]

I also suggest drawing a convex hull for "roadway" intersections (primary, secondary, tertiary, residential, etc).

This is already done. The major road intersections have a round dot and their convex hull is darker (or more saturated, large alpha) than the minor road intersections, which have a smaller square dot and a brighter (less saturated, small alpha) convex hull. Major road intersections are defined as those, that have at least one way with a category, that does not belong to the categories of minor roads (pedestrian, track, footpath, path, cycleway, bridleway).

I made this version for visualization purposes only, so we can discuss issues using real-world examples.

[vvoovv]

I meant crossing nodes where ALL ways are not minor roads (pedestrian, track, footpath, path, cycleway, bridleway).

[polarkernel]

I meant crossing nodes where ALL ways are not minor roads (pedestrian, track, footpath, path, cycleway, bridleway).

I see. I made now a version, that produces an additional category of intersections, which have only major roads. These are plotted as circle dots and their convex hull color is fully saturated with alpha=1.

For many clusters, there are only two such major intersections, so it is not possible to draw a convex hull. These are then connected by a thick line:

The version with this feature is committed.

[vvoovv]

I made now a version, that produces an additional category of intersections, which have only major roads.

To be more precise, a crossing node can be shared by minor ways, but it must be shared by at least 3 major ways.

[polarkernel]

To be more precise, a crossing node can be shared by minor ways, but it must be shared by at least 3 major ways.

OK, committed version adapted.

[vvoovv]

To be more precise, a crossing node can be shared by minor ways, but it must be shared by at least 3 major ways.

Sorry. That sentence was not formulated precisely again. I meant at least 3 major way-segments instead of at least 3 major ways.

[polarkernel]

Sorry. That sentence was not formulated precisely again. I meant at least 3 major way-segments instead of at least 3 major ways.

To be precise, here are the rules of the current implementation. For each node that is an intersection, the sum wayCount of all incoming and outgoing ways is determined. The number of ways belonging to the categories pedestrian, track, footpath, path, cycleway or bridleway, is determined as nrOfLow and the rest as nrOfMajor.

The following intersection types are assigned according to the number of different categories:

• If all wayCount ways belong to the categories pedestrian, track, footpath, path, cycleway or bridleway (wayCount==nrOfLow), the intersection node is classified as low intersection (plotted as small rectangle).
• If at least three of these ways are major roads (nrOfMajor> 2), the intersection node is classified as major intersection (plotted as large ring).
• If there are major roads, but less than three (nrOfMajor< 3), the intersection node is classified as main intersection (plotted as large dot).

[vvoovv]

For each node that is an intersection, the sum wayCount of all incoming and outgoing ways is determined.

Did you mean way-segments or ways?

If a node is located at the middle of a way and there are no other ways that share the node, then wayCount=1 and waySegmentCount=2

[polarkernel]

Did you mean way-segments or ways?

I mean way-sections. I have never used the concept of ways that go through an intersection, because it can be ambiguous. If there are multiple way-sections at an intersection, it is not always clear, which ones belong to the same way.

[vvoovv]

I'll think what to do with the filets in that context.

Created #83

[vvoovv]

The question now is, where do the bundles (Bundle class) and the intersections (Intersection class) start? What about crosswalks? Do they belong to the bundles, and what about their intersections?

An intersection is created by finding the intersection points of the roadway borders. So a crosswalk is a part of the roadway.

But there is a notable exception: a diagonal crossing or a pedestrian scramble :

It's a part of a intersection and painted right on it. I suggest ignoring the minor ways (footways, crossways) when generating the intersections. Then the footways located within the intersection polygon can be filtered out.

I added the new OSM file _streets/tokyoshibuya.osm which contains the famous Shibuya Crossing.

[polarkernel]

I added the new OSM file _streets/tokyoshibuya.osm which contains the famous Shibuya Crossing.

Very interesting crossing, but will be difficult to construct correctly:

[vvoovv]

Very interesting crossing, but will be difficult to construct correctly:

What will be the problems?

[vvoovv]

_streets/berlin_highways_withoutintersections.osm, the intersection at the middle.

The darkest polygon includes the leftmost crossing node. However it is not the part of the intesection. Would it possible to fine tune the algorithm, so that the leftmost crossing node won't be included to the darkest intersection polygon?

[polarkernel]

Would it possible to fine tune the algorithm, so that the leftmost crossing node won't be included to the darkest intersection polygon?

The current algorithm is still quite simple. It is based on a density-based spatial clustering algorithm (DBSCAN), using the endpoints of all equidistant way-sections. It does not yet distinguish between the three intersection categories we have defined. An important parameter is the distance between cluster points, which is currently simply the Euclidean distance. This version has a tendency to connect clusters (or points) along strings of neighboring points, which seems to have happened in your example.

The algorithm has a lot of potential for improvement and control, which I will tweak as I get a better understanding of what the output should look like. For example, the distance can be weighted by the intersection types. There is also a hierarchical version of this algorithm. There is a lot of literature on this method of clustering.

[polarkernel]

What will be the problems?

This is only a first intuitive impression.

[vvoovv]

I think the current results are enough to start generating some output in Blender. I suggest starting to generate the output using the new data structures. The algorithm to generate the intersections of bundles can be tweaked afterwards.

[polarkernel]

I suggest starting to generate the output using the new data structures.

OK. First steps might be:

• Definition of the attributes and methods needed by the addon for rendering the Section class.
• Introduction of some kind of getStyle function, callable within StreetGenerator, which processes way-sections (Section class) by their PML style.
• I don't remember: Are still PML modifications required for getStyle? If so, where are they defined?
• Definition of a container for Section objects in the WayManager, which is filled by StreetGenerator as soon as they are fully processed and ready for rendering.

[vvoovv]

• Definition of the attributes and methods needed by the addon for rendering the Section class.

https://github.com/prochitecture/blosm/issues/72#issuecomment-1905748582

• Introduction of some kind of getStyle function, callable within StreetGenerator, which processes way-sections (Section class) by their PML style.

I will create an initial implementation of getStyle function and related PML.

• I don't remember: Are still PML modifications required for getStyle? If so, where are they defined?

Modifications aren't needed right now. Generation of the streets is more important at the moment.

• Definition of a container for Section objects in the WayManager, which is filled by StreetGenerator as soon as they are fully processed and ready for rendering.

What is the difference between this item and the first one?

[polarkernel]

What is the difference between this item and the first one?

When a Section is processed by getStyle, it is returned to StreetGenerator for further processing. It can then be trimmed to fit to intersection connectors, be integrated into bundles, be invalidated, because it is too short, and the more. After that, it has to be transferred back to the addon for rendering, which requires some kind of container, I think, for this transport.

Of course, without the intersection areas, the streets can be rendered directly in the getStyle call for now.

[vvoovv]

The function getStyle will return a dictionary with the default values for some attributes (e.g. the number of lanes) and the values of the attributes that are not available in OSM at all (e.g. fillet radii),

What if the same instance of the class Section is used throughout the whole processing pipeline? It can have an attribute valid which is set to False if the section became invalidated.

[polarkernel]

The function getStyle will return a dictionary with the default values for some attributes (e.g. the number of lanes) and the values of the attributes that are not available in OSM at all (e.g. fillet radii),

OK, I will wait for your proposition.

What if the same instance of the class Section is used throughout the whole processing pipeline? It can have an attribute valid which is set to False if the section became invalidated.

Then the instance of this Section is the data in an edge of the instance self.waymap (WayMap class) in StreetGenerator. No problem, then, WayMap is the container I am talking about. If later the renderer needs to access a specific instance of Section, an adequate access function will have to be provided by WayMap and access to self.waymap has to be provided.

But we don't have to define that right now. We can work our way forward one step at a time.

[vvoovv]

Are the attributes default, lane, doubleRoadsideWidth from the dictionary wayCategoryProps at _way/wayproperties.py used now in the code?

[polarkernel]

Are the attributes default, lane, doubleRoadsideWidth from the dictionary wayCategoryProps at way/way_properties.py used now in the code?

lane is currently used in the function estimateWayWidths() as an estimate for the lane width, when no width is given in the section tags. The other two (default and doubleRoadsideWidth) are presently not used. The only additional attribute in the code is nrLanes, which is actually used in the function lanePattern(). Both functions are in the file _way/wayproperties.py.

[vvoovv]

I will create an initial implementation of getStyle function and related PML.

It has been committed.

An additional parameter will be required in the command line: --assetsDir /path/to/folder/assets.

Currently --assetsDir should point to the unpacked attached file. It contains the PML file for streets: assets/default/style/street/main.pml.

I introduced the class Street in the module way.item.street. It should serve as a sort of wrapper for everything between two intersections.

The StreetGenerator constructor now accepts two parameters: styleStore and setStyle. They are set as the attributes of the StreetGenerator instance in the constructor. An example code can be found between the lines 129 and 138 of the file _blosm/setup/mplblosm.py.

A way should be proposed to access the first and the last items in a Street. In the example code the first item (Section) is accessed via the attribute start.

Once a Street will be populated with items, I will finish assigning the style for all street items in the method Street.setStyle(..) of the module way.item.street.

I renamed way.items to way.item.

I added the parent class Item for all classes in way.item. However I think not all classes in way.item require the parent class Item.

[vvoovv]

The default number of lanes and other default parameters are defined in the style block street for a given style. The attribute totalNumLanes and, where appropriate, totalNumLanesOneway are used for that.

[polarkernel]

The default number of lanes and other default parameters are defined in the style block street for a given style. The attribute totalNumLanes and, where appropriate, totalNumLanesOneway are used for that.

During my research on lane patterns, I found that finding the number of lanes is a difficult process. In the current code, this is done by finding a lane pattern in the function lanePattern() (line 171 in /way/way_properties.py). The attributes isOneWay, totalLanes, forwardLanes, backwardLanes and bothLanes are then derived from this pattern in the Section class. Additionally, the instances of the classes SymLane and SideLane will be constructed using this pattern. Now, I do not understand how to combine this pattern with the default parameters from the style block. What is totalNumLanesOneway?

There are two things we should not forget at this time:

• The meaning of the number of lanes is different for left-hand traffic and for right-hand traffic. In lanePattern(), this is taken into account by a function argument. But we will need a parameter (from the way-manager?), that gives this attribute.
• We have defined some categories, that do not halve the way width, when they are tagged as one-way. This is implemented in the tuple halveNrLanesIfOneWay (line 44 in /way/way_properties.py). I don't know if you want to keep these exceptions.

[polarkernel]

A way should be proposed to access the first and the last items in a Street. In the example code the first item (Section) is accessed via the attribute start.

Something like this?

Forward (start to end) should correspond to drawing direction of OSM operator, to keep correct lane directions.

[vvoovv]

Now, I do not understand how to combine this pattern with the default parameters from the style block.

The values from the dictionary wayCategoryProps should be replaced by the values from a PML style block.

What is totalNumLanesOneway?

If street.getStyleBlockAttr("totalNumLanesOneway") returns None, then the total number of lanes is equal to street.getStyleBlockAttr("totalNumLanes") if it's a oneway section.

[vvoovv]

I created a temporary repository to host PML styles for streets: https://github.com/prochitecture/assets

[vvoovv]

Something like this?

Exactly.

Forward (start to end) should correspond to drawing direction of OSM operator, to keep correct lane directions.

There can be more complex cases.

A Street can also contain a Bundle.

Also consider the following Street: Section - SymLane - Section. Those sections could have the opposite directions.