AADT of closest major road

[unbrother]

I have managed to create some of the variables except for the aadt of closest major road, mentioned in the original methodology. I know there this would be something similar to what was done in the original code shared with me by @mem48 but can't find the correct steps to do it. @Robinlovelace

[mem48]

I don't think I got that far in the example I wrote. I summose you could measure the straight distance to all the major road traffic counts and pick the closets, but it might we nice to have a method that related to the road network.

[mem48]

IT might be better to start with the traffic count points and work out the roads that they serve, rather than the otherway round.

[mem48]

You might alos consider vonoi polygons sf::st_voronoi() the function is a little tricky to use and doesn't maintain the point order, but is very useful

[mem48]

postcodes_mp <- sf::st_combine(postcodes_sub)
voronoi <- try(sf::st_voronoi(postcodes_mp, envelope = oa_sub$geometry), silent = TRUE)
if("try-error" %in% class(voronoi)){
 ennv <- sf::st_buffer(oa_sub, 1000)
 voronoi <- sf::st_voronoi(postcodes_mp, envelope = ennv$geometry)
}
voronoi <- sf::st_collection_extract(voronoi)
sf::st_crs(voronoi) <- 27700
voronoi <- sf::st_as_sf(data.frame(id = 1:length(voronoi), geometry = voronoi))
voronoi <- sf::st_join(voronoi, postcodes_sub)

[unbrother]

I have managed to implement the voronoi polygons by but did not based them in the existing points data but only points from major roads. This creates a somewhat useful layer but I'm stuck in how to assign AADT from major roads to minor roads. This is the last step before the linear regression in the original SQL code.

# filter points inside bounds points_major <- st_line_sample(osm_major, 1, density = 0.5, type = "regular", sample = NULL) qtm(points_major) points_major <- sf::st_combine(points_major) voronoi <- try(sf::st_voronoi(points_major, envelope = bounds$geometry), silent = TRUE) #if("try-error" %in% class(voronoi)){ # ennv <- sf::st_buffer(bounds, 2000) # voronoi <- sf::st_voronoi(points_mp, envelope = ennv$geometry) #} voronoi <- sf::st_collection_extract(voronoi) sf::st_crs(voronoi) <- 27700 voronoi <- sf::st_as_sf(data.frame(id = 1:length(voronoi), geometry = voronoi)) voronoi <- sf::st_join(voronoi, points) # qtm(voronoi) + qtm(osm_urban, lines.col = "aadt", lines.lwd = 1)

Also, I am not sure how to add a full chunk of code. I had to mark line by line.

[mem48]

You can do a code chunk with three `

start of code ```

code goes here

end of code ```

[mem48]

COuld you put some screenshots of the layers? It hard to visualise from the code alone

[unbrother]

First I used points only from major roads, which I think I could reduce based on sections or something else, but I am still thinking about how to do it.

# filter points inside bounds
points_major <- st_line_sample(osm_major, 1, density = 0.5, type = "regular", sample = NULL)
qtm(points_major)

Then I ran the voronoi code, works without the try-error so I commented that.

points_major <- sf::st_combine(points_major)
voronoi <- try(sf::st_voronoi(points_major, envelope = bounds$geometry), silent = TRUE)
#if("try-error" %in% class(voronoi)){
# ennv <- sf::st_buffer(bounds, 2000)
# voronoi <- sf::st_voronoi(points_mp, envelope = ennv$geometry)
#}
voronoi <- sf::st_collection_extract(voronoi)
sf::st_crs(voronoi) <- 27700
voronoi <- sf::st_as_sf(data.frame(id = 1:length(voronoi), geometry = voronoi))
voronoi <- sf::st_join(voronoi, st_as_sf(points_major))
#
qtm(voronoi) +
qtm(osm_urban, lines.col = "aadt", lines.lwd = 1)

I am getting the voronoi without the bounds so they are extending outside of the Isle. Also, I tried to visualize the major roads by AADT to see if that could be a criteria to select points. On the Morley model, they created areas of interest which are "distinct networks of minor roads only accessible to each other without a major road being crossed" and created the polygons this way.

[mem48]

Sorry for the slow reply, been very busy.

THe problem I see with using voronoi polygons is that you might associate traffic with a major road that is nearby but not connected.

When I was trying to do this I started by idetifiying all the places the major and minor roads intersected. That defiens the point where traffica can lear the major road network and join the minor road network. You then want to associate the minro roads with the closesr one of those points on the road network. This is what I was using dodgr for.

[unbrother]

I think I managed to get this by replicating the code for nearest neighbours for references.

osm_cents <- st_coordinates(st_centroid(osm_urban))

nn_aadt = RANN::nn2(osm_cents, k = 20) ### TRY TO FIX THIS LOOP ###

osm_urban$aadt_major = osm_urban$aadt

## Needs to run three times

for(k in 1:2){
  for(i in 1:nrow(osm_urban)){
    if(is.na(osm_urban$aadt[i])){
      for(j in 2:20){
        idx <- nn_aadt$nn.idx[i,j]
        if(osm_urban$roadtype[idx] == osm_urban$roadtype[i]){
          if(!is.na(osm_urban$aadt[idx])){
            osm_urban$aadt[i] <- osm_urban$aadt[idx]
            break
          }
        }
      }
    }
  }
}

After repeating the loop three times, most of the network ends with AADT from major roads, after reviewing this, they all have the data from the nearest road until it can not find one. I am now struggling with making the for loop repeat those three times (which might be more for some other network) until it has no more roads to assign. I ran the loop 5 times but after the third one, no more roads seemed to change. Questions Is this correct? If it is. How can I adapt the code for it to run until the network is fully assigned?

[unbrother]

Just realized that I might need to restrict that only aadt from major roads is pasted into minor, because some minor roads data were copied. I am not sure how to do this inside the for loop.

[mem48]

I suppose you can just run the k loop more times. Should you be using j in 1:20 rather than 2:20? In the references code, the 1st nearest neighbor is itself, but in this case, I don't think that is true. The map you should certainly doesn't look right. Perhaps you can make the major roads bigger to make it clear where the course of the values is.

You still have the disadvantage of measuring euclidian distance rather than network distance to desice the nearest neighbour.

[mem48]

Also, I've marked your poster and sent it to @Robinlovelace for second marking, so hopefully, we will have it back to you soon.

[unbrother]

@mem48 and @Robinlovelace , I was finally able to assign AADT from nearest major roads.

I believe this assignment makes more sense. My next problem is road importance. Will open another issue for that if this can be closed

[Robinlovelace]

This looks like great progress!

[unbrother]

Hi @mem48 and @Robinlovelace, After reviewing the results, the AADT copied is not right, as whenever a minor road has previous data, it only copies from itself instead of from major roads. I am trying by first assigning all major roads and then copying from minor, but still most of the minor roads take data from other minors.

# Finish major roads (populate only major road)

for(k in 1:50){
  for(i in 1:nrow(osm_urban)){
    if(osm_urban$roadtype[i] == "major"){
      if(is.na(osm_urban$aadt_major[i])){
        for(j in 1:ncol(nn_aadt$nn.idx)){
         idx <- nn_aadt$nn.idx[i,j]
          if(osm_urban$roadtype[idx] == "major"){
           if(!is.na(osm_urban$aadt_major[idx])){
             osm_urban$aadt_major[i] <- osm_urban$aadt_major[idx]
             break
            }
          }
        }
      }
    }
  }
}

# Populate minor roads from nearest major road

for(k in 1:50){
  for(i in 1:nrow(osm_urban)){
    if(osm_urban$roadtype[i] == "minor"){
        for(j in 1:ncol(nn_aadt$nn.idx)){
          idx <- nn_aadt$nn.idx[i,j]
          if(osm_urban$roadtype[idx] == "major"){
            if(!is.na(osm_urban$aadt_major[idx])){
              osm_urban$aadt_major[i] <- osm_urban$aadt_major[idx]
              break
          }
        }
      }
    }
  }
}

qtm(osm_urban, lines.col = "aadt_major", lines.lwd = 3)

The first 4 rows seem to have been assigned correctly but the remaining rows all have the same value for aadt and aadt_,major.

I am uploading a new commit with the full code. There is something I am not understanding inside the loops and the indexing. Because I believe the second for loop is first making sure it only populates minor roads but I am not sure how to make sure it only takes data from major roads.

My next approach might be to create another index for the second loop, but I am not sure if that would work or I would be just going in circles.

[mem48]

@unbrother I've just committed two new files haven taken a look at this.

01_getdata.R is a rework of your getdata.R and downloads and cleans the data

02_matchmajorminor.R does the interesting matching

I'll talk you through it on Monday but the key stages are:

1) Use dodgr distances to measure the distance from the centroid of each minor road to each junction point between a major and a minor road

2) FIlter to the shortest distance, to find the closest junction point to each road

3) COnstruct an OD matrix from minor road centroids to junction points, with a flow of 1 and pass this to dodgrflows aggregate. THis gives a map of the road network is a count of how important each road is

4) Match the junction points to the nearst major road, and give the junction points the AADT value of that road

5) take the AADT value from the junction point and assign it to the corresponding minor road

Still left to do is to combine the AADT and importance score

You'll notice that there are a lot of NAs they have come from major roads with NA as their AADT so perhaps we need to better clean the major road data.

[mem48]

@unbrother following https://github.com/ATFutures/dodgr/issues/137 can you try a few things:

1. remove dodgr and reinstall, just to make sure you are on the correct version

   remove.packages("dodgr")
   install.packages("dodgr")

2. add the argument norm_sums = FALSE to dodgr_flows_aggregate to see if that gives you better results

[unbrother]

@mem48 Coming back to this part, I believe since AADT from nearest major road and importance are two different variables for the original model, the part of the code that just assigns AADT from the nearest junction should still be used. I am getting the following results, which seem correct to be used as a variable I named aadt_maj

I think it would be better for the next part (the importance) to use a dataset that already contains this variable, so next I am trying to consolidate the new osm_aadt_maj (which only contains data for minor roads) with the original osm (which has data for ALL roads). I am trying via a join but do not get the expected result, which is the original data but with the new added column.

osm_aadt_maj <- st_join(osm, osm_minor_mod, by = c("osm_id", "osm_id"))

Instead, I get doubled variables, tagged as osm.x, osm.y, ....

My idea is that this "osm2" object already contains the variable so that I am able to replicate the model (before trying to assign my own AADT with a different method).

[mem48]

St_join is a spatial join, it would be easier to rbind osm_major and osm_minor. The row column names need to match but you can easily add some empty columns if needed.