search

chairemobilite / transition

Transition is a modern new approach to transit planning. It's a web application to model, simulate and plan public transit and alternative transportation.
http://transition.city
MIT License
20 stars 13 forks source link

We should not have to duplicate postgis spatial indexes #921

Open greenscientist opened 2 months ago

greenscientist commented 2 months ago

The GenericPlaceCollection class create its own spatial index to store points in memory. We should not have to do that kind of operations in Typescript, we have a powerful spatial database optimized for that!

tahini commented 2 months ago

We'll have to list all operations that make use of this local spatial index and move them closer to the database (ie in the backend if they are potentially run in the frontend) so they can benefit from it. Then we can remove this index.

tahini commented 2 months ago

Adding this to the Path refactoring milestone, as it implies revisiting those geographic collection to make them smaller (and thus maybe not sufficient for the required location operations)

tahini commented 2 months ago

One note though, the local indexes are sometimes used to index data that is not yet in the database. For example, when importing GTFS stops, when we aggregate new stops with existing ones, we use this local index, then add new ones or update some after moving their position a little to the centroid of the aggregated nodes.

Only at the end of all node import do we save them in the database. The stop aggregation algorithm will need to be revisited if we want to use postgis.

Proposal:

  1. Aggregate locally the new nodes to import first (using either kdbush still, or some function of turf)
  2. Aggregate the resulting aggregated GTFS stops with the existing nodes (using postgis)

The advantage of this approach is that is prioritizes aggregating the stops being imported rather than the currently existing nodes, which may have been imported with different aggregation distances and thus may separate two related nodes, which are close by in the current import, but may be aggregated to different original nodes.

greenscientist commented 2 months ago

We might want to revisit the whole agregation concept. Maybe we could save all node data as "raw" node without any agregation. We could then maybe have an addition layer, "NodeAgregated" that we use when we really need to do operation on agreagated nodes.

tahini commented 2 months ago

We might want to revisit the whole agregation concept. Maybe we could save all node data as "raw" node without any agregation. We could then maybe have an addition layer, "NodeAgregated" that we use when we really need to do operation on agreagated nodes.

True, because if you import a stop with a radius of 0, you don't want the next import, with radius of 60, to move your precious node! Which is what is currently happening...