[GN][streets] First steps towards streets and intersections using GN

[polarkernel]

The results from the construction of trimmed way-sections and intersection polygons have to be adapted for the use with objects based on geometry nodes. Let's discuss here the details of this task.

[polarkernel]

Copied from a PM of @vvoovv :

Sorry, I'm not at all familiar with Blender curves. Does that mean that the width (the radius of a control point) has to be set at construction time?

Yes.

Alternatively, the width can be set as a parameter of a GN-setup. In this case, the width is applied to all splines in the Blender object

Another alternative is to create a separate Blender object for each spline. Then every parameter of a way-section can be controlled by a GN-parameter.

I am ready to continue there. Maybe it's best if we take small steps there and start first with standard intersection areas, before the complex ones. Just tell me, what I can do.

I need a trimmed way-section if it connects two intersections. I will make a curve spline out of the trimmed way-section and apply a GN-setup to a Blender object containing the splines that represent way-sections of the same type.

I need polygons for intersections. The problem is how to map sides of a polygon and way-sections that are attached to them. Perhaps like this: Let v1, v2, v3, v4 be the vertices that form a polygon. Then it has the sides v1_v2, v2_v3, v3_v4, v4_v1. The way-section ws1 is attached to the side v1_v2, ws2 to v2_v3, ws3 to v3_v4, ws4 to v4_v1. So we'll have a list of way-sections (ws1, ws2, ws3, ws4)

[polarkernel]

Alternatively, the width can be set as a parameter of a GN-setup. In this case, the width is applied to all splines in the Blender object. Another alternative is to create a separate Blender object for each spline. Then every parameter of a way-section can be controlled by a GN-parameter.

Another alternative could be somehow in the middle of the two propositions: Create GN-based material for every road category and its number of lanes, and use it for all way-sections of the same type. This could also include different textures for the carriage way and the sidewalks, if any.

[vvoovv]

Suppose we have N road categories (primary, tertiary, residential, etc) and L is the maximum number of lanes. Did you suggest to create N*L GN-setups?

I thought it is the number of lanes that matters at the moment. I don't know how to distinguish visually a tertiary road with 2 lanes from a residential road with 2 lanes.

[polarkernel]

Did you suggest to create N*L GN-setups?

Sure, the only difference for these categories is possibly the number of lanes, but I had also footways or cycleways in mind.

[polarkernel]

I need a trimmed way-section if it connects two intersections. I will make a curve spline out of the trimmed way-section and apply a GN-setup to a Blender object containing the splines that represent way-sections of the same type. I need polygons for intersections. The problem is how to map sides of a polygon and way-sections that are attached to them.

Let me discuss this based on the structures that already exist in my experimental code (no need that they remain, but it makes it easier to understand).

Way-sections

All way-sections are stored in a dictionary with an identification number as key, let me call it wsID. Their class WaySection contains a polyline with the vertices of the way-section's centerline in the order they have been drawn by the OSM operator. This in the hope that one-way sections have a known direction. The start of the way section is called Source and abbreviated with S and the end is called Target, abbreviated with T. The terms originate from the network graph and I kept them for consistency.

These way-sections include two values for width, leftWidthand rightWidth, so that turning lanes may be represented, where the left width may be larger than the right one. Maybe we should keep that for the moment until we know, how these can be constructed using GNs. Left and right means the side when looking from source to target. These parameters make also sense for way-sections with different numbers of lanes for both directions (based on the tags lanes:forward and lanes:backward, which I did not yet implement).

The attributes trimSand trimTkeep the information on where to trim the centerline on both sides. Trimming the centerline currently takes place later, but this is not important for the moment.

Intersection polygons

When I proposed to start with simple intersections, I wanted to exclude clusters of intersection, but not intersections like this one (with or without fillets):

Intersection polygons are just lists of vertices in counter-clockwise order. In my experiments, I did not join them with the way-section polygons. I don't know if it will be the intention to connect these polygons with the GN-setup. The connectors are the straight lines between two vertices at the position, where the way-sections are connected. These are computed based on the width of the way-section and the alignment of their centerline. It is important that the spline will end there perpendicular to the connector, as the centerline already does.

Possible interface

Based on your proposition and on the facts mentioned above, I propose the following interface:

A list or a dictionary of all way-sections, accessible by an ID or the list index, where every way-section contains:

• the trimmed polyline (in forward direction)
• the category of the way-section
• the left and right width relative to the centerline
• possibly additional tags?

A list of intersection polygons, where each polygon contains:

• A list of its vertices, ordered counter-clockwise
• A connector description (see below)

This connector description is a list of all connectors, where every entry contains:

• the index of the first vertex of the connector in the list of its vertices *)
• the ID of the way-section connected there
• the end of this way-section connected here (S or T)

*) It should be omitted during the construction of the polygon that a connector exists between the last and the first vertex of the polygon. Then, the second vertex is always the next one after the first.

[vvoovv]

That sounds reasonable. Let's stick to you proposal for the moment.

I haven't yet started with a GN-setup for intersections polygons, so I am not yet sure what data structures will be need for the intersection polygons.

• possibly additional tags?

If the number of lanes are already used in the processing pipeline, it can be added to the interface. Otherwise I'll extract them from the OSM tags.

[polarkernel]

If the number of lanes are already used in the processing pipeline, it can be added to the interface. Otherwise I'll extract them from the OSM tags.

It is important that the left and right width of a way-section at both ends is known at the time when the intersection area is constructed, so that its connectors are correctly aligned. Until now, I just used default widths for the carriage ways and the tag lanes, if available, to get these widths (see _way/wayproperties.py). To get the true widths, including the correct number of lanes and including side-walks and the like, is terribly difficult and makes me belly ache. A/B Street's project osm2lanes employed a large team for more than a year, and is not yet ready.

Otherwise I'll extract them from the OSM tags.

Yes, I think it will be good in any case to add a reference to the tags to the interface.

Do you have an idea on how we should organize the new task of creating streets and intersections using GN? Should we eventually start a new branch for that, once we see how the interface should look like? The branch _streetsintersections contains an enormous amount of ballast due to the many experiments I have done. I would like to refactor and clean it and use only the necessary code required for our collaboration.

[vvoovv]

I suggest to start a new branch from the existing branch dev and copy necessary pieces of code from the branch _streetsintersections.

[polarkernel]

I suggest to start a new branch from the existing branch dev and copy necessary pieces of code from the branch streets_intersections.

Created new branch _streets_forgn, forked from dev. I will prepare there the code towards my proposition of the interface. The final interface can be defined later.

[polarkernel]

I refactored the branch _streets_forgn and added a fragment of the class StreetsForGN for the construction of streets and intersection areas. The setup file _mplblosm.py and some related classes and definitions have been "modernized" using the changes we made on the way between dev and _streetsintersections.

All command line arguments used in the branch dev (--buildings, --highways, --detectFeatures, --classifyFacades", --simplifyPolygons", --facadeVisibility, --showAssoc, --showIDs, --showFeatures, --restoreFeature) do still work, while a new argument --gnStreets has been added to activate the action in the method do() of StreetsForGN.

[vvoovv]

I'm using the argument --gnStreets but I can't see truncated way-segments and intersection on the resulting chart. I tried it on _berlin_karl_marxallee.osm. How can I see them on the chart and get access to them in the code?

[polarkernel]

I'm using the argument --gnStreets but I can't see truncated way-segments and intersection on the resulting chart. I tried it on berlin_karl_marx_allee.osm. How can I see them on the chart and get access to them in the code?

Sorry, I am not so fast. The committed code only creates the networkGraph and the sectionNetwork and renders these results. I am working on the creation of the intersection areas and the trimmed way-sections using the proposed interface proposition, but this will still take a moment.

[polarkernel]

A first version of the construction of intersection areas and the trimmed way-sections using the argument --gnStreets is committed.

The method do() of the class StreetsForGN in _gnstreets.py creates in several steps the required results. After the call to the method createOutputForGN (line 28), a dictionary self.waySectionLines holds the centerlines of the trimmed way-sections. The keys of this dictionary are IDs (numbers), while its values are instances of the class WaySection_gn. This class is defined in a separate file _output_forGN.py, so that it can easily be changed if required. The attributes of this class are as follows:

        centerline      # The trimmed centerline in forward direction (first vertex is start.
                        # and last is end). A Python list of vertices of type mathutils.Vector.
        category        # The category of the way-section.
        startWidths     # The left and right width at its start, seen relative to the direction
                        # of the centerline. A tuple of floats.
        endWidths       # The left and right width at its end, seen relative to the direction
                        # of the centerline. A tuple of floats.
        tags            # The OSM tags of the way-section.

The intersection areas are stored in the Python list self.intersectionAreas of StreetsForGN. This is a list of instances of the class IntersectionPoly_gn, also defined in the file _output_forGN.py. The attributes of this class are:

        polygon         # The vertices of the polygon in counter-clockwise order. A Python
                        # list of vertices of type mathutils.Vector.
        connectors      # The connectors (class Connector_gn) of this polygon to the way-sections.
                        # A dictionary of tuples, where the key is the ID of the corresponding
                        # way-section key in the dictionary of WaySection_gn. The tuple has two
                        # elements, <indx> and <end>. <indx> is the first index in the intersection
                        # polygon for this connector, the second index is then <indx>+1. <end> is
                        # the type 'S' for the start or 'E' for the end of the WaySection_gn
                        # connected here.

Currently, this list includes both, intersection areas and transitions (areas that connect two way-sections of different width). Should it be required to separate them, maybe because different textures should be used, it would be easy to create separate lists of identical structures.

The type of the intersection area can be selected by the value self.useFillet (line 24 of file _gnstreets.py). If True, the areas are constructed using fillets and else just with edges. For the moment, I propose to use them without fillets, because then less intersections between nearby intersection areas are produced.

Finally, the last method plotOutput of StreetsForGN paints the result using mpl.

I will prepare a next version, where I try to merge intersecting areas of way-intersections (maybe I should call them conflicting ares). Some time ago, I wrote some code for a union operator for polygons that works much faster than the one from pyGEOS. The main problem I have there is to retrieve the positions of the connectors, so this may take a while. However, the interface described above will not change, so that first tests are already possible.

[vvoovv]

I have a minor note about naming. I think the prefix or suffix GN isn't needed. There is nothing specific to Geometry Nodes in the processing pipeline.

[vvoovv]

I'll revert _setup/mplblosm.py to its original state and then insert the required setup code for StreetsForGN.

[polarkernel]

I'll revert setup/mpl_blosm.py to its original state and then insert the required setup code for StreetsForGN.

It might be good if you already rename StreetsForGN to avoid having the suffix GN.

[vvoovv]

It might be good if you already rename StreetsForGN to avoid having the suffix GN.

Yes, I'll do it.

[vvoovv]

File _facade_visibility/manhattan01.osm.

Why are roads so narrow (the width is only 1m)?

[polarkernel]

Why are roads so narrow (the width is only 1m)?

There was a bug in estimateWayWidth (fixed now) and a strange tagging of pedestrian ways as one-way (?). I don't know if pedestrians in Manhattan walk only in one direction. For ways that have no tags for the number of lanes, the tag oneway: 'yes' halves the default width in the method estimateWayWidth. I excluded this now for pedestrian ways. Maybe this should be done for a whole group of way categories.

[polarkernel]

It might be good if you already rename StreetsForGN to avoid having the suffix GN.

I would like to suggest renaming more things. According to the command line tag --generateStreets, the following should be renamed:

The file _action/gnstreets.py --> _action/generatestreets.py The class StreetsForGN --> StreetGenerator

Then, by reasons I do no more remember, the managerpassed as argument to the do() method of the class StreetsForGN could not be used to get the ways. Already in the branch _streetsintersections I had to use

        wayManager = self.app.managersById["ways"]

Should this eventually be corrected in _setup/mplblosm.py, so that the correct manager is passed to do()?

[vvoovv]

For ways that have no tags for the number of lanes, the tag oneway: 'yes' halves the default width in the method estimateWayWidth. I excluded this now for pedestrian ways. Maybe this should be done for a whole group of way categories.

I suggest not to halve the width for all categories of carriageways with oneway=yes. I guess they were originally designed as two ways roads and then transformed to one way roads to optimize traffic.

[vvoovv]

According to the command line tag --generateStreets, the following should be renamed:

Done.

Should this eventually be corrected in _setup/mplblosm.py, so that the correct manager is passed to do()?

Everything has been fixed in the latest commit. I removed the lines wayManager = self.app.managersById["ways"] in _action/generatestreets.py.

[polarkernel]

Everything has been fixed in the latest commit. I removed the lines wayManager = self.app.managersById["ways"] in action/generate_streets.py.

Thanks!!

[vvoovv]

There were some dependencies in the code that prevent running it in Blender:

• imports of matplotlib: I moved them inside the related methods and functions
• unused import of KDTree from scipy.spatial: the related line was deleted

[polarkernel]

I am near to finish the merge of conflicting intersection areas (that appear almost only for areas using fillets). I will then commit quite a large refactored part. At the moment, however, I am irritated by a very strange behavior of mathutils.Vector:

sequence[79]
Vector((284.3341979980469, -236.76954650878906))
sequence[80]
Vector((284.334228515625, -236.76954650878906))
sequence[79]==sequence[80]
True
sequence[79][0]==sequence[80][0]
False
sequence[79][1]==sequence[80][1]
True

It appears not only using the self built version of mathutils for Python 3.10, but also in older versions. I found this as a reason for a bug in my own polygon union operator. Maybe an effect of the single precision floats in mathutils.Vector? Don't know how to handle that, maybe I will replace Vector by a tuple in the operator temporarily.

[vvoovv]

Very irritating problem!

Don't know how to handle that

How about using the condition

sequence[79][0]==sequence[80][0] and sequence[79][1]==sequence[80][1]

instead of the condition

sequence[79]==sequence[80]

[polarkernel]

How about using the condition

The example shows only the reason for the bug. But in my implementation, for instance the test

            if p in verts:

returns True, when it shouldn't. This observation has severely undermined my trust in mathutils. Perhaps other strange problems were also due to something like this.

[polarkernel]

I have committed a largely refactored version that implements now the merge of conflicting intersection areas. Additionally, several bugs have been fixed and some renaming is done to remove the indication to GNs. In detail, the following changes have been made:

Bug fixes:

Some intersection areas provided duplicate vertices, fixed now. Some intersection areas produced self intersections, only partly fixed, the others are mostly caught for the moment, I hope.

Merge of conflicting intersection areas

Works for most of the intersections. Intersection areas with fillets may be used now. There are still rare troubles in some data files that I will have to fix later.

Renaming

The following classes are renamed: WaySection_gn --> TrimmedWaySection IntersectionPoly_gn --> IntersectionArea Both have been moved from the file _output_forGN.py to _generatestreets.py. The file _generatestreets.py is removed. The method self.createOutputForGN() of StreetGenerator is renamed to self.createOutput().

Obsolete

For the current development state, the library pygeos is not required, however, I left it because we might need it later.

[vvoovv]

            if p in verts:

returns True, when it shouldn't.

Can you provide an example for that?

[polarkernel]

Can you provide an example for that?

        v1 = Vector((284.3341979980469, -236.76954650878906))
        v2 = Vector((284.334228515625, -236.76954650878906))
        v3 = Vector((283.999999, -225.7777777706))
        print('v1 == v2: ', v1 == v2)
        verts = [v2, v3]
        if v1 in verts:
            print('v1 is in list')
        else:
            print('v1 is not in list')

Output:

        v1 == v2:  True
        v1 is in list

For the moment, I solved by using tuples. There is not much computation in BoolPolyOps.py. Like this, it works now.

[vvoovv]

I see.

I created an issue in the repository of mathutils. I hope it will be fixed.

[vvoovv]

I am going to create a separate Blender object out of each street section and apply a GN-setup to each street section. This will allow direct editing of each of them via the parameters of the GN-setup.

The width of a carriageway will be set via a parameter in a GN-setup instead of the radius of a control point.

[vvoovv]

The attributes of this class are as follows:

        centerline

I suggest including the number of lanes to the attributes of the class TrimmedWaySection since you use them anyway to calculate the width of carriageways.

[polarkernel]

I suggest including the number of lanes to the attributes of the class TrimmedWaySection since you use them anyway to calculate the width of carriageways.

Do you need the total number of lanes or separated into number of lanes right and left?

[vvoovv]

I'd prefer separated (left and right)

[polarkernel]

I'd prefer separated (left and right)

Sorry when I bother you again. When I started the implementation, I found that the meaning of left and right is not yet clear. Do we mean left and right of the centerline, or do we mean the driving direction? For a one-way carriageway of the Karl-Marx-Allee for instance, the pair (left, right) would be (1½,1½) for the first case and (0,3) for the other. I assume the second makes more sense.

[vvoovv]

When I started the implementation, I found that the meaning of left and right is not yet clear.

For two-ways carriageways: ( the number of lanes on the left side relative to the direction of the centerline, the number of lanes on the right side relative to the direction of the centerline )

For one-way carriageways only one number is needed.

[vvoovv]

I moved the matplotlib rendering code code from _action/generatestreets.py to mpl/renderer/streets.py.

[polarkernel]

For two-ways carriageways: ( the number of lanes on the left side relative to the direction of the centerline, the number of lanes on the right side relative to the direction of the centerline ) For one-way carriageways only one number is needed.

Implemented and committed. TrimmedWaySection contains now an attribute nrOfLanes, which is a tuple that contains either the (left,right) number of lanes relative to the centerline for two-ways carriageways or one single integer for one-way carriageways.

[polarkernel]

I have committed a new version, where a special case with short ways between intersections is processed to get a correct intersection area now. The interface did not change. I also added the base class Renderer to StreetRenderer, because the program complained about a missing finalize() before.

[vvoovv]

I commented out the line 116 in _way/wayintersections.py since it contained calls to matplotlib.

[vvoovv]

Some preliminary results:

Sidewalks overlap currently. Changing the width of a carriageway segment leads to the inconsistency with the intersection polygon.

[vvoovv]

Carriageways in _osm_extracts/facade_visibility/manhattan01.osm are too narrow (only 2m). Do you halve the width for oneway=yes?

[polarkernel]

I commented out the line 116 in way/way_intersections.py since it contained calls to matplotlib.

Thanks and sorry.

[polarkernel]

Carriageways in osm_extracts/facade_visibility/manhattan_01.osm are too narrow (only 2m). Do you halve the width for oneway=yes?

In a new commit, I added a set of way categories whose width is not halved for one-way streets (keepWidthIfOneWay, line 42 in _way/wayproperties.py). You may adapt it to your needs. Please feel also free to change the widths given in the table wayCategoryProps (line 7). 'default' is the width, when the number of lanes is unknown, and 'lane' the width of one lane else. The radius of the fillet at intersections, if used, is given in 'radius'. I made this table using satellite images, but just to have something to start, without a thorough examination.

[vvoovv]

'default' is the width, when the number of lanes is unknown, and 'lane' the width of one lane else.

I suggest to use the default number of lanes instead of the default width if the number of lanes is unknown. I also suggest to calculate the width of a carriageway as follows: width = nrLanes * laneWidth + 2*roadsideWidth

It would be also required store in the dictionary wayCategoryProps the value of 2*roadsideWidth. I can enter the values of defaultNrLanes and doubleRoadsideWidth to the dictionary wayCategoryProps if you agree.

[vvoovv]

I need the number of lanes to apply the appropriate texture to the carriageway.

[polarkernel]

It would be also required store in the dictionary wayCategoryProps the value of 2*roadsideWidth. I can enter the values of defaultNrLanes and doubleRoadsideWidth to the dictionary wayCategoryProps if you agree.

Sure I agree. Please create the adapted dictionary, and then I will immediately refactor the remaining computations.

[vvoovv]

I refactored _way/wayproperties.py. However now I am getting an error. I guess that's because some ways are too broad.

If I remove props['doubleRoadsideWidth'] in the line 89, the error disappears.

[vvoovv]

However now I am getting an error. I guess that's because some ways are too broad.

The test file was _osm_extracts/facade_visibility/manhattan01.osm.