HSLdevcom / jore4

Public transport registry
European Union Public License 1.2
4 stars 2 forks source link

Fix Jore3-Importer export logic for conflicting validity periods between lines and routes #1659

Closed jarkkoka closed 8 months ago

jarkkoka commented 8 months ago

Fix jore3-importer logic related to database constraint violations caused by validity period inconsistencies between transformed lines and routes in order to import more routes into Jore4.

In the Jore3 data model, the line itself (the network_lines table in the internal database of jore3-importer) has no validity period. Instead, the line-specific headers (network_line_headers table) have validity periods that do not overlap. Originally, it was thought that a separate line instance would be created for each line header into the Jore4 database. However, it appears that Jore3 route directions (network_route_directions table) may overlap with multiple line headers. This causes a transformation problem, because in Jore4 the validity period of a route MUST NOT extend further into the past or future than the line to which the route belongs.

Basically, there are two options to solve the problem:

  1. For each line in Jore3 (and all related line headers), create only single line in Jore4.
  2. Group and join line headers together by validity periods in such a way that no route overlaps multiple line instances transformed to Jore4.

ATM, we have chosen the option (2) to import as much (version) data as possible into Jore4. In the line export SQL, we group the line headers into temporally continuous clusters. The last line header of each cluster is given a Jore4 ID that can be referenced by routes.