od_jitter error

[Hussein-Mahfouz]

I am running od_jitter::jitter() but keep running into an error. I get the same error when I run the example provided in the repo. I was going to raise in issue in odjitter, but I see that the project is depreciated.

od_jittered = jitter(od, zones, subpoints = road_network)
Scraped 7 zones from /tmp/RtmpYSxjfY/zones.geojson                                                                                                                    
Scraped 5073 subpoints from /tmp/RtmpYSxjfY/subpoints.geojson
Scraped 5073 subpoints from /tmp/RtmpYSxjfY/subpoints.geojson
Disaggregating OD data
Wrote /tmp/RtmpYSxjfY/od_jittered.geojson
Error: Cannot open "/tmp/RtmpYSxjfY/od_jittered.geojson"; The source could be corrupt or not supported. See `st_drivers()` for a list of supported formats.
In addition: Warning message:
In CPL_read_ogr(dsn, layer, query, as.character(options), quiet,  :
  GDAL Error 1: GDALOpen() called on /tmp/RtmpYSxjfY/od_jittered.geojson recursively

@Robinlovelace are you aware of this error? do you get a similar error when you run the example?

[Robinlovelace]

I have seen various errors before, not sure what the cause of this is.

Follow-up questions that could help diagnose the source of the issue and find a fix:

1. Does /tmp/RtmpYSxjfY/od_jittered.geojson exist and if so what's in it?
2. Do the basic examples in odjitter::jitter() work?

In some cases I've seen recently the lack of subpoints in zones have been a cause but I don't have any sense of what the cause is in this case ATM.

[Hussein-Mahfouz]

Thanks Robin. I will explore the subpoints issue - perhaps there are no feasible solutions ( I am specifying subpoints, disaggregation_threshold, min_distance_meters, weight_key_origins, weight_key_destinations - see here)

1. /tmp/RtmpYSxjfY/od_jittered.geojson does exist but I think it's corrupt

   
   > file.exists("/tmp/Rtmp857f2J/od_jittered.geojson")
   [1] TRUE
   > head(readLines("/tmp/Rtmp857f2J/od_jittered.geojson"), 10)
   + }
   [1] "{\"type\":\"FeatureCollection\", \"features\":["                                                                                                                                                                                                                                                                                                                   
   [2] "{\"geometry\":{\"coordinates\":[[-1.39708332871112,53.92874996492413],[-1.38874999541112,53.92874996492413]],\"type\":\"LineString\"},\"properties\":{\"Destination\":\"jitterE02002331\",\"Origin\":\"jitterE02002330\",\"geometry\":\"c(-1.40008023067835, -1.34744921305028, 53.9294384236734, 53.9228592546686)\",\"total_flow\":60.5},\"type\":\"Feature\"}," 
   [3] "{\"geometry\":{\"coordinates\":[[-1.39708332871112,53.92874996492413],[-1.322083329011121,53.92874996492413]],\"type\":\"LineString\"},\"properties\":{\"Destination\":\"jitterE02002331\",\"Origin\":\"jitterE02002330\",\"geometry\":\"c(-1.40008023067835, -1.34744921305028, 53.9294384236734, 53.9228592546686)\",\"total_flow\":60.5},\"type\":\"Feature\"},"
   [4] "{\"geometry\":{\"coordinates\":[[-1.40541666201112,53.920416631624136],[-1.70541666081112,53.912083298324134]],\"type\":\"LineString\"},\"properties\":{\"Destination\":\"jitterE02002332\",\"Origin\":\"jitterE02002330\",\"geometry\":\"c(-1.40008023067835, -1.7042900018817, 53.9294384236734, 53.9111158431453)\",\"total_flow\":5.0},\"type\":\"Feature\"}," 
   [5] "{\"geometry\":{\"coordinates\":[[-1.39708332871112,53.93708329822413],[-1.70541666081112,53.90374996502413]],\"type\":\"LineString\"},\"properties\":{\"Destination\":\"jitterE02002332\",\"Origin\":\"jitterE02002330\",\"geometry\":\"c(-1.40008023067835, -1.7042900018817, 53.9294384236734, 53.9111158431453)\",\"total_flow\":2.0},\"type\":\"Feature\"},"   
   [6] "{\"geometry\":{\"coordinates\":[[-1.40541666201112,53.92874996492413],[-1.713749994111121,53.912083298324134]],\"type\":\"LineString\"},\"properties\":{\"Destination\":\"jitterE02002332\",\"Origin\":\"jitterE02002330\",\"geometry\":\"c(-1.40008023067835, -1.7042900018817, 53.9294384236734, 53.9111158431453)\",\"total_flow\":4.0},\"type\":\"Feature\"}," 
   [7] "{\"geometry\":{\"coordinates\":[[-1.39708332871112,53.92874996492413],[-1.65541666101112,53.895416631724125]],\"type\":\"LineString\"},\"properties\":{\"Destination\":\"jitterE02002333\",\"Origin\":\"jitterE02002330\",\"geometry\":\"c(-1.40008023067835, -1.68782474109376, 53.9294384236734, 53.9007518576381)\",\"total_flow\":6.0},\"type\":\"Feature\"}," 
   [8] "{\"geometry\":{\"coordinates\":[[-1.38874999541112,53.93708329822413],[-1.663749994311121,53.895416631724125]],\"type\":\"LineString\"},\"properties\":{\"Destination\":\"jitterE02002333\",\"Origin\":\"jitterE02002330\",\"geometry\":\"c(-1.40008023067835, -1.68782474109376, 53.9294384236734, 53.9007518576381)\",\"total_flow\":3.0},\"type\":\"Feature\"},"
   [9] "{\"geometry\":{\"coordinates\":[[-1.40541666201112,53.92874996492413],[-1.72208332741112,53.895416631724125]],\"type\":\"LineString\"},\"properties\":{\"Destination\":\"jitterE02002333\",\"Origin\":\"jitterE02002330\",\"geometry\":\"c(-1.40008023067835, -1.68782474109376, 53.9294384236734, 53.9007518576381)\",\"total_flow\":8.0},\"type\":\"Feature\"}," 
   [10] "{\"geometry\":{\"coordinates\":[[-1.38874999541112,53.93708329822413],[-1.33041666231112,53.862083298524134]],\"type\":\"LineString\"},\"properties\":{\"Destination\":\"jitterE02002334\",\"Origin\":\"jitterE02002330\",\"geometry\":\"c(-1.40008023067835, -1.35752759942759, 53.9294384236734, 53.8828600749283)\",\"total_flow\":92.0},\"type\":\"Feature\"},"

od_jittered <- st_read("/tmp/Rtmp857f2J/od_jittered.geojson")

Cannot open "/tmp/Rtmp857f2J/od_jittered.geojson"; The source could be corrupt or not supported. See st_drivers() for a list of supported formats. Warning message: In CPL_read_ogr(dsn, layer, query, as.character(options), quiet, : GDAL Error 1: GDALOpen() called on /tmp/Rtmp857f2J/od_jittered.geojson recursively

2. The basic example runs in a fresh r session, but when I run my example and then try to run  the basic example, it gives the same error (possibly due to the tempdir not being empty)

[Robinlovelace]

Really odd. A good next step in such a situation could be to contrain the possible causes through a process of elimination. If you try jittering a subset of OD pairs, a subset of zones, a subset of subpoints, and different combinations, that could help. If you have minimal example datasets with the inputs that are failing as .geojson files, an make them available somewhere, I can have a look, to at least see if I can reproduce the issue.

[Robinlovelace]

On more follow-up question: which version of odjitter are you using, the Rust crate?

[Robinlovelace]

You could try this:

git clone https://github.com/dabreegster/odjitter && cd odjitter && cargo build --release && cp ./target/release/odjitter /usr/local/bin/

Source: https://github.com/dabreegster/odjitter/issues/52

[Hussein-Mahfouz]

I was using the rust crate yes. I think it might be that there are no feasible solutions that match my disaggregation_threshold constraint (possibly because there aren't enough subpoints). I don't know what the behaviour is if that's the case. I will try the elimination approach you suggested, update you, and prepare a minimal dataset if it persists

[Robinlovelace]

From memory they're should be an argument allowing repeat sampling of points..

[Hussein-Mahfouz]

The problem was due to there not being a feasible solution. Example:

• disaggregation_threshold = 100,
• OD pair E02002330 - E02002376 that has flows = 200, but there is only 1 subpoint in E02002330

The function fails as there are no points to split the 200 onto. I think this should raise an informative warning in odjitter. As a workaround, I've written a small function to split an sf point layer (each pint is split into n_points, and the numeric column is split equally between the n_points). See split_points: https://github.com/Hussein-Mahfouz/drt-potential/pull/31/commits/845fdc4220e3bb65cb322c95eecd349c8286e657

[Robinlovelace]

Great stuff, thanks for the update on this one by the sounds of it. Feel free to open, and potentially fix at some point, an issue in the odjitter issue tracker.