Failure to read road attribute from routerDb within Lua file

[jerryfaust]

Hello.

I want to modify the factor_and_speed result in a custom Lua file, but I am unable to get the necessary attribute from the routerDb. In fact, I can't seem to get any values from the routerDb, and I don't know of any way to debug through the Lua file to see what's happening at run-time (is there a way?).

This is a Shapefile-based routerDb, and everything else works fine - routes and directions are properly generated based on this Lua file. So my road names for the directions are properly retrieved from the meta_whitelist, but I cannot seem to get any values from fields based on the profile_whitelist. I have mimicked almost exactly the whitelists and factor_and_speed from the Lua file from the Dutch national road sample. Here is the top part of my file (excluding the 'directions' portion):

name = "shape.car"

-- whitelists for profile and meta
profile_whitelist = {
    "IS_CLOSED",
    "streetname"
}

meta_whitelist = {
    "road_name"
}

-- profile definitions linking a function to a profile
profiles = {
    {
        name = "",
        function_name = "factor_and_speed",
        metric = "time"
    },
    { 
        name = "shortest",
        function_name = "factor_and_speed",
        metric = "distance"
    }
}
-- the main function turning attributes into a factor_and_speed and a tag whitelist
function factor_and_speed (attributes, result)

    result.speed = 50    -- Km/h
    result.direction = 0
    result.canstop = true
    result.attributes_to_keep = {}

    -- get data attributes
    local IS_CLOSED = attributes.IS_CLOSED
    local streetname = attributes.streetname
    -- is the road closed ?
    if IS_CLOSED then
        if IS_CLOSED == "T" then
            result.speed = 0
            result.direction = 0
            result.canstop = true
            result.attributes_to_keep.IS_CLOSED = true --.IS_CLOSED = IS_CLOSED
        end
    -- else
    --     result.speed = 0
    end
    -- if streetname == "PENDLETON" then
        -- result.speed = 0
        -- result.direction = 0
        -- result.canstop = true
        -- result.attributes_to_keep.streetname = true
    -- end
end

Some comments:

1. My original intent was to check the value of IS_CLOSED. This was originally a boolean field, but every sample I see of Lua is reading values as strings, so just to make sure, I changed my field to a string value "T" or "F". Does the result.attributes field hold other data types?
2. My best guess for behavior is that I only need to add the field to the profile_whitelist in order to be able to read its value within the 'attributes' parameter of factor_and_speed. Is there something else I need to do load the values?
3. I can simply change the initial speed setting from 50 to 0 to verify that I can no longer set endpoints and generate routes. So I'm confident that this specific Lua file is governing my run-time behavior.
4. Suspecting that I'm not fetching anything (getting back nil), I added the "if IS_CLOSED then" condition along with the else result.speed = 0, in order to verify that I'm never even getting into the "if IS_CLOSED == "T" block of code.
5. As a side-note, I see some samples that set "result.attributes_to_keep. = true" and some samples that set "result.attributes_to_keep. = ". Is that field looking for a boolean or a field value? (So far it hasn't mattered since I'm never getting to that code).

Thank you for any guidance. Regards. Jerry.

[xivk]

First of all I think these are missing:

https://github.com/itinero/geo/blob/master/samples/Sample.Shape/car.lua#L4

The columns identifying the ID's of the vertices, making the shapefile into something that describes topology.

1. My original intent was to check the value of IS_CLOSED. This was originally a boolean field, but every sample I see of Lua is reading values as strings, so just to make sure, I changed my field to a string value "T" or "F". Does the result.attributes field hold other data types?

No just string key-value pairs.

2. My best guess for behavior is that I only need to add the field to the profile_whitelist in order to be able to read its value within the 'attributes' parameter of factor_and_speed. Is there something else I need to do load the values?

No you have done fine I think.

3. I can simply change the initial speed setting from 50 to 0 to verify that I can no longer set endpoints and generate routes. So I'm confident that this specific Lua file is governing my run-time behavior.

There is a logging function we've built in to do debugging. I think you can do

itinero.log("Some message here!")

For reference, here is the source of that function: https://github.com/itinero/routing/blob/develop/src/Itinero/Profiles/Lua/ItineroLib/ItineroModule.cs#L70

4. Suspecting that I'm not fetching anything (getting back nil), I added the "if IS_CLOSED then" condition along with the else result.speed = 0, in order to verify that I'm never even getting into the "if IS_CLOSED == "T" block of code.

I'm not 100% sure but try setting the source_vertex and target_vertex columns.

5. As a side-note, I see some samples that set "result.attributes_to_keep. = true" and some samples that set "result.attributes_to_keep. = ". Is that field looking for a boolean or a field value? (So far it hasn't mattered since I'm never getting to that code).

That's a good question, I checked the code and it's only using the keys of that 'dictionary' so it doesn't matter really:

https://github.com/itinero/routing/blob/develop/src/Itinero/Profiles/DynamicVehicle.cs#L253

[jerryfaust]

Thank you, @xivk , I am now able to act upon our IS_CLOSED attribute (The logging helped. There was nothing wrong with the script, but instead with some Itinero code I had modified for other reasons :-(

I was reviewing Customizable Contraction Hierarchies (#92, #208, et. al.), even downloading the Paper published on the ACM site, since a client of mine would like to be able to close roads at run-time. It is rather complex, and it will not be trivial to incorporate this feature.

I then saw #159, in which you mention a proposed feature to allow the editing of edge meta-data (at run-time). My thought was to add the attribute (IS_CLOSED), defaulting to False initially, then editing that attribute at run-time to close specific roads. I believe this would require using a non-Contracted network, but I think the trade-off would be acceptable (the largest network we might have is a US County).

If I am able to get approval to pursue this development, is there any direction you could give me, or prior notes/considerations/Issues related to this option?

Regards. Jerry.

[xivk]

I just did a few commits today to make this exact scenario possible. You can do this now, update a specific edge's data after the routerdb has been loaded or preprocessed and do routing on it while doing this.

The only thing that's not possible is to use CH in this case, so routing will be slow. There are some performance improvements on the plain dykstra implementation that we haven't considered doing because the CH is there. What I'm saying is there is some room for improvement in the regular dykstra-based routing.

If you do this I would recommend adding edge profiles, even if they don't exist yet, for each edge for both types, closed and not closed. That way you only need to update 2bytes of data per edge when changing status from closed to not closed and vice versa. The routerdb can stay in the same state and updating can be very fast.

Also, you can calculate a CH for the scenario not closed and only use dykstra when it hits one of the closed edges. This give you speed in case there are no closures along the route with a marginal increase in calculation time.

It would be great if there is a way some of your work could benefit the project. If you can give feedback like this that's fine but some code that flows back into the project is also welcome... You can also buy a couple hours of consulting time where we review your work or support the project with a donation.

[jerryfaust]

Hello. I've been building/running the 'master' branch up until now. I presume that I need the 'develop' branch (or similar) to get your latest commits.

1. I downloaded the 'develop' branch, but I'm having trouble building it, with issues related to netstandard20. Is there build documentation available?
2. Additionally, the 'develop' branch does not appear to have the IO.Shape components. Is there a different branch I should get which includes these?

Thank you. Jerry.

[jerryfaust]

Ok. Quick follow-up.

1. I got past the build issue for now by just nixing the netstandard20 target.
2. I would still like to confirm the branch I should be working against. I assume 'develop'. But I'm not sure why the Itinero.Geo and Itinero.IO.Shape projects are not included. Kind Regards. Jerry.

[jerryfaust]

Update: I am able to close roads and regenerate routes without having to rebuild the network - albeit in a very rudimentary way. But for now this is just a proof-of-concept, to know it can be done. I've attached some notes, with a question at the end. Regards. Jerry. Itinero Issue 212.docx

[xivk]

The Itinero.IO.Shape code was moved to this repo:

https://github.com/itinero/geo

[xivk]

About the 13k edges, you have to look at the profiles as 'types' of edges. Try not to include any data about the edges like id's or names in the profile.

[alecava58]

@jerryfaust I need to close one or more edge on the fly too. I like you solution and I'll give it a try as soon as I can.

Recently I tried to solve using restrictionsDb. In short: create a restrictionsDb with the vertices you want to close (you can select a bounding box of vertices if you want to close an area). once loaded the routingDb attach the restrictionsDb to it and give it a name. when you don't want the restrictions any more you can remove the restrictionsDb from the routingDb without reloading the network.

I know that restrictionsDb has not be done for this purpose but it seems to be working.

[jerryfaust]

@alecava58

Yes, early on, I tried using restrictions as well, and maybe you're having more success than I was. But as you know, they are intended for turn restrictions, and my understanding is that they may inadvertently block routing through a vertex that is not intended to be closed (but I am by no means an expert on this).

My current philosophy involves re-assigning an edges' profile at run-time, but I still have a few things to work through to make sure it cooperates with existing functionality. Also note that using this method, you cannot use a contracted network, so it will not be as fast as it otherwise could be. But with smaller networks (city or county based), I believe it is an acceptable trade-off.

After I work some things out, I will submit my proposed changes. I'm not sure that they will be acceptable for formal inclusion in Itinero, but I think it will be a working solution for some.

Regards. Jerry.

[jerryfaust]

@xivk

As an alternative to what I was trying before, I have now developed a means of maintaining a simple list of Closed Roads (as Edge IDs). Currently stored in the Router class, it is completely independent of the edge profiles and the Lua code. The RouterDb does not have to be regenerated. I simply check against this list within the Step method of the Dykstra implementations, bypassing those edges that are tagged as closed.

Although there may be more work to be done to fill in any blanks, I think it is a viable option for smaller non-Contracted networks. I would like to Push my branch and submit a Pull request for review, but I don't have permission. It is possible to get this permission? I have already signed a contributor agreement.

Thank you. Jerry.

[xivk]

Just FYI:

The Itinero.IO.Shape and Itinero.Geo code was moved back here after the stable release of NTS.