cityofaustin / atd-vz-data

The technology that powers the City of Austin's Vision Zero program
https://visionzero.austin.gov/viewer/
11 stars 3 forks source link

Skip CRIS import record updates if a key column is missing a value #1437

Closed mddilley closed 5 months ago

mddilley commented 5 months ago

Associated issues

TLDR: Older records in the person and primary person tables that are missing the prsn_occpnt_pos_id are breaking the SQL that we use to query and update records that receive updates from CRIS. This is one of the columns that we use to check if a person row is unique.

Testing

URL to test: Local

Steps to test:

  1. Ask me for a zip file that I created that contains csvs with only one crash and its related person, primary person, and unit records that can replicate the original bug 🪄🪱 You can see our current values for the people records in the test csv with the following query (which some of these appear to be duplicates):
    SELECT
    atc.crash_id,
    atc.crash_date,
    atp.prsn_nbr,
    atp.unit_nbr,
    atp.prsn_type_id,
    atp.prsn_occpnt_pos_id
    FROM
    atd_txdot_person atp
    LEFT JOIN atd_txdot_crashes atc ON atp.crash_id = atc.crash_id
    WHERE
    atc.crash_id = 14003860;
  2. See readme for local testing instructions
  3. Run the import on the zip and you should see the person records that are missing the prsn_occpnt_pos_id column value logged as skipped

Releasing this: We should see the production image of the CR3 import update after merging this to master. GH action code


Ship list