Add setup.R

[robinlovelace-ate]

Latest here FYI, will look for a quick fix. Heads-up that this worked with the R so I think od2net could be a bit more resilient here:

sudo docker run -v $(pwd):/app ghcr.io/urban-analytics-technology-platform/od2net:main /app/config.json
Using config from /app/config.json:
{
  "requests": {
    "description": "Test data for SchoolRoutes project.",
    "pattern": {
      "BetweenZones": {
        "zones_path": "zones.geojson",
        "csv_path": "od.csv"
      }
    },
    "origins_path": "buildings.geojson",
    "destinations_path": "buildings.geojson"
  },
  "cost": "Distance",
  "uptake": "Identity",
  "lts": "BikeOttawa",
  "elevation_geotiff": "elevation.tif"
}

## everything
#### Load network
Trying to load network from /app/intermediate/network.bin
#### Load network took 40.522558ms
#### Loading or generating requests
###### Loading origins
Loading points from /app/input/buildings.geojson
###### Loading origins took 56.341751ms
###### Loading destinations
Loading points from /app/input/buildings.geojson
###### Loading destinations took 57.006865ms
Got 59,088 origins and 59,088 destination
###### Loading zones from /app/input/zones.geojson
###### Loading zones from /app/input/zones.geojson took 485.807µs
###### Matching points to zones
0 zones have no matching origin points. Using centroid instead.
0 zones have no matching destination points. Using centroid instead.
###### Matching points to zones took 17.102089ms
###### Generating requests from /app/input/od.csv
Error: Unknown zone S02001576

[robinlovelace-ate]

Oh, I know: it's because the schools haven't been added as destinations!

[robinlovelace-ate]

Reproducible error after fixes in commit above:

docker run -v $(pwd):/app ghcr.io/urban-analytics-technology-platform/od2net:main /app/config.json
Using config from /app/config.json:
{
  "requests": {
    "description": "Test data for SchoolRoutes project.",
    "pattern": {
      "BetweenZones": {
        "zones_path": "zones.geojson",
        "csv_path": "od.csv"
      }
    },
    "origins_path": "buildings.geojson",
    "destinations_path": "destinations.geojson"
  },
  "cost": "Distance",
  "uptake": "Identity",
  "lts": "BikeOttawa",
  "elevation_geotiff": "elevation.tif"
}

## everything
#### Load network
Trying to load network from /app/intermediate/network.bin
#### Load network took 37.763019ms
#### Loading or generating requests
###### Loading origins
Loading points from /app/input/buildings.geojson
###### Loading origins took 56.9863ms
###### Loading destinations
Loading points from /app/input/destinations.geojson
###### Loading destinations took 97.019µs
Got 59,088 origins and 63 destination
###### Loading zones from /app/input/zones.geojson
###### Loading zones from /app/input/zones.geojson took 456.264µs
###### Matching points to zones
0 zones have no matching origin points. Using centroid instead.
###### Matching points to zones took 24.949962ms
WARNING: Dropping timer during block Loading or generating requests. Probably crashing.
Error: Some zones have no matching destination points: ["E01013448", "E01013403", "E01033068", "E01013354", "E01013336", "E01013360", "E01013439", "E01033067", "E01013380", "E01013394", "E01013338", "E01013351", "E01013423", "E01013427", "E01013372", "E01013444", "E01013392", "E01013408", "E01013370", "E01013397", "E01013384", "E01013342", "E01013385", "E01013361", "E01013405", "E01013366", "E01013400", "E01013389", "E01013429", "E01013401", "E01013440", "E01013386", "E01013435", "E01013434", "E01034762", "E01013399", "E01013344", "E01013411", "E01013398", "E01013420", "E01013339", "E01013388", "E01013412", "E01033070", "E01013432", "E01013341", "E01013424", "E01013349", "E01013449", "E01013446", "E01013373", "E01033069", "E01013347", "E01013418", "E01013441", "E01013356", "E01013371", "E01013445", "E01013343", "E01013353", "E01013382", "E01013337", "E01013357", "E01013359", "E01013436", "E01013346", "E01013376"]

[dabreegster]

Generating requests from /app/input/od.csv

Error: Unknown zone S02001576

This kind of error is deliberate; better to be very clear about a problem in the input data.

Some zones have no matching destination points:

I can double-check your inputs and sanity check this is indeed true, but I think the error is describing reality here. Do you want to fallback to zone centroid or something else?

[robinlovelace-ate]

Generating requests from /app/input/od.csv

Error: Unknown zone S02001576

This kind of error is deliberate; better to be very clear about a problem in the input data.

That was from the previous error, not present in latest error message, right?

[dabreegster]

That was from the previous error, not present in latest error message, right?

Yes

About the zones without destinations: it really looks true to me:

[robinlovelace-ate]

About the zones without destinations: it really looks true to me:

Yes, these are schools. There is not one in every zone, right?

[robinlovelace-ate]

I recall we had a similar issue with odjitter and I created a work-around with a small zone around each school. Should I try that here? I suspect upstream fix would be better if my hunch is right.

[dabreegster]

I recall we had a similar issue with odjitter and I created a work-around with a small zone around each school. Should I try that here? I suspect upstream fix would be better if my hunch is right.

Maybe it's fine that some zones have no destinations, because later in the OD data, there are no flows that ever go to a zone without a school. I could relax the error message here into a warning, and fail requests later that try to go to a zone with no points. Or we could use the zone centroid as a fallback. Creating new zones sounds more complex to me.

Is the shape of your OD data "origin zone, destination school/point"? If so, use ZoneToPoint instead of ZoneToZone. See liverpool and york examples in od2net

[robinlovelace-ate]

Is the shape of your OD data "origin zone, destination school/point"? If so, use ZoneToPoint instead of ZoneToZone. See liverpool and york examples in od2net

Yes, that should fix it, thanks!

[robinlovelace-ate]

Making headway. Quick question for you @dabreegster, does od2net expect the origins and destinations to be wrapped in quotes, e.g.

from,to,count
E01013335,"121266",6.372101583408054

not

from,to,count
E01013335,121266,6.372101583408054

[dabreegster]

It tries to interpret from and to as strings. Not sure offhand what the Rust CSV reader allows here. Are you having a problem? If so, wrapping in quotes should fix it.

[robinlovelace-ate]

Thought so and easy to wrap. Bingo, just in time for live demo at 3pm!

[robinlovelace-ate]

Tests pass so merging.

[robinlovelace-ate]

And had a basic review + repro test from Sam (many thanks).