Worker-side single point accessibility

[abyrd]

This calculates accessibility on the worker for single-point requests, and returns it in the JSON block at the end of the binary travel time grid (alongside the scenario application warnings). As usual, axis order is [destinationPointSet][percentile][cutoff], and this should support up to ten destination point sets of varying dimensions. There should be 121 values in each innermost array, for each cutoff from 0 to 120. Array index 5 would be a cutoff of 5 minutes, so all opportunities reachable in less than (5 * 60) seconds. This is "less than" rather than <= for consistency with our old definition: cutoffs are applied on the worker at one-second resolution, but on the UI travel times were floored, so a travel time of 4 minutes represented all times up to but not including 5:00 minutes.

{ 
  accessibility: [
    [
      [6,7,8,...],
      [5,6,7,...],
      [4,5,6,...],
      [3,4,5,...],
      [2,3,4,...]
    ],
    [
      [17,19,21,...],
      [15,16,17,...],
      [13,14,15,...],
      [11,12,13,...],
      [9,10,11,...]
    ]
  ],
  scenarioApplicationWarnings: [],
  scenarioApplicationInfo: []
}

This is rough around the edges but can serve as a starting point to initiate work on the UI side. The main differences in UI-backend communication are the need to send a destinationPointSetId with single point requests (when one is selected) and the ability to use the accessibility property of the response instead of calculating the accessibility within the UI.

Due to the application of distance decay functions, we may want to make the shift to non-integer accessibility values (especially for cases where there are relatively few opportunities across the whole study area). So ideally, interpret the accessibility property as if it contained floating point numbers, even though it currently contains integers.

This also introduces a wrapper class for mapping between indexes of grids of different dimensions. This is necessary because we want to display isochrones for the full region even if an opportunity data set covers a smaller geographic area, and if a user requests a small search area we don't want to search out to the full extent of the opportunity datasets (which could be much larger).

This transform class is being applied in the same way for single point and regional tasks, so should also allow regional analyses with multiple grids of different sizes, though I haven't tested that yet.

Applying the distance decay function to every travel time at every destination at each of 120 cutoffs does slow down propagation, but optimizations of that process should not affect the external interface and can be deferred.

[abyrd]

This is working for single-point requests and can be tested with conveyal/analysis-ui#1210. For regional analyses there is a chicken and egg problem: loadAndValidateDestinationPointSets uses getWebMercatorExtents(), but that method needs the dimensions of all the pointsets, which assumes they're already loaded. I'll need to fix that.

[abyrd]

This is now in a usable state, but is still marked "draft" because it might need a couple of rounds of refactoring or optimization, which will be facilitated by having a UI branch that can communicate with it.

[abyrd]

Actually @trevorgerhardt have you seen https://github.com/conveyal/analysis-ui/pull/1210 ? I believe that's what we need to do to enable worker-side accessibility calculation. At least requesting it, of course displaying it is another change. But I guess that UI PR is only a stopgap (which I can use for testing) if we are planning to push the opportunity selector down into the primary/comparison search settings in the UI.

[trevorgerhardt]

@abyrd will this break if the UI sends a destinationPointSetIds array to older worker versions?

[abyrd]

@abyrd will this break if the UI sends a destinationPointSetIds array to older worker versions?

The intent is that the UI can send the same structure to the backend no matter what worker is selected, and the backend will perform any renormalization (thus avoiding needing multiple ui-to-backend request versions). But there are some "work in progress" blocks of code in this PR, which I will clean up during testing against the corresponding front end PR conveyal/analysis-ui#1210 including testing that it works with both older and newer workers.

When that's confirmed and cleaned up I'll remove the draft PR status.

[ansoncfit]

With v5.10.0-25-gb73b853 and the related front-end changes, the accessibility results differ depending on whether the isochrone is fetched before or after the opportunity dataset is selected (i.e., whether the accessibility result is calculated in the front-end or the worker). That's understandable given the conditional logic in the front-end to calculate accessibility when it receives a null accessibility result but could be disconcerting to users.

[abyrd]

My understanding was that if the dataset is changed after accessibility is calculated on the worker, the results would become invalid and be grayed out until the request is sent again. If/when accessibility is being computed UI side (with an older worker, or if we choose to do so for the step decay function) then the accessibility chart could be updated, but I think this would just be perceived as a bonus (don't need to click the "compute" button to clear the grayed/invalidated results). I wouldn't expect this to confuse users.

[ansoncfit]

To summarize our discussion, it appears there is an off-by-one difference in how the accessibility result array slots are interpreted.

In this example, the red curve is calculated by the front-end and the blue curve is calculated by the worker:

For consistency with the front-end (and I think the regional results), the single-point accessibility result entry with index 0 should correspond to a cutoff value of 60 seconds (i.e. the number of opportunities reachable in less than 60 seconds). The isochrone above, corresponding to the minimum value of the travel time slider (1), should correspond to all the places reachable in less than 60 seconds.

[abyrd]

@ansoncfit if I follow the links above about comments, it just takes me to the top of the "files" page rather than a specific line of a file. Can you provide the filenames and line numbers?

[ansoncfit]

@abyrd Both in TravelTimeReducer: Javadoc TODOs at lines 41-51, and comment about Grids at 239-241