Real-Time Traffic Incident Reports Discovery on Metadata

[dianamartin]

From this meeting we've determined we should do our due diligence on this dataset

• We should reach out to CTECC contacts
• Find out what certain fields means Issue Reported
  • get a data dictionary? Chandy M. and Sherry S.
• Update metadata fields at the same time we update the data viz per CTM request
• Provide data viz narrative for the data viz

Goal: So we can provide accurate description of what the data is and isn't, provide supplementary information and where to find data resources.

Thu 1/25/24 - meeting with Apps and Data Science

Originally posted by @KaroEngstrom in https://github.com/cityofaustin/atd-data-tech/issues/15411#issuecomment-1910701892

[dianamartin]

1/26/2024

• [x] Emailed CTECC IT contacts (who we think are contacts)

Hi Chandy/Sherry, My name is Diana, I’m one of the IT Supervisors in the Transportation and Public Works Department (TPW). I work in the shared IT division, Data and Technology Services that supports TPW. We have a dataset out there on the Open Data Portal that originates from a Public Safety database, the “Real-Time Traffic Incident Reports”.

History of the Dataset

• the dataset was based on an RSS feed published by CTM from the Oracle DB until Spring 2023; at that point we began to pull the data from the CTM Oracle data connection directly. • this dataset was audited while CTM published a story page for it (now deprecated) with the same content as our current story page, with the exception of them using a 5 minute RSS feed widget over our data table at the top of the page

Previous CTM story page:

The CTM Open Data team has reached out to us regarding some actionable items that came from the 2023 audit report from the Office of the Auditor. In the report, it was recommended to the City Manager that departments interpret the most desired data with visualizations and/or data stories, in addition to providing the raw data. Therefore, departments are required to provide stories and visualizations for the most-accessed datasets.

We’re interested in learning more about the data we publish as we are now de facto data owners of this dataset because we publish an aggregated format. I am wondering if it this data looks familiar to you and if you can point me in the right direction on who to speak to get some clarification on some fields. . We are trying to be good data stewards and seek out information to help populate this critical metadata.

I’m interested in getting a data dictionary of the coded values for the “Issue Reported” field.

If you’re not the right contacts, please feel free to let me know who I might contact to get these answers.

[dianamartin]

2/23/24

Notes from 2/9/24 Meeting

Participants:

• DTS: Diana, Karo, Charlie
• CTM-CTECC: Chandy M. - IT Manager
  • Most public safety applications
  • CADD - Computer Aided Dispatch
• Kendrick M. - IT App Consultant
  • Application Services Team (lead) CADD
• Oleg K. - IT Manager
  • Public Safety Enterprise Services team
  • SQL Servers

Real-Time Traffic Incident Reports

• From CADD, public safety agencies from CTECC
• APD
• Travis County Sheriff’s
• Airport

911 Call for service, responder response

Agency’s data - who sees their data,

CADD - CJIS/HIPPA

• mandated by federal government

Agencies that are sharing from CADD

• APD, AFD, Travis County Sheriffs, ABIA

Public Safety History

• 2003: Decided to share with citizens of Austin

• Traffic related incidents

  • Backup traffic

• Problem Nature:

  • Dispatcher/Officer in field: code it
    • traffic hazard
    • collision leaving the scene
    • vehicle fire

• Published Data: put into the CADD system

  • Address: citizen said address or MDC
    • Change change (block range), they may have actual address
  • Status: Active

• Production database —> CADD Data Warehouse

Data Pipeline

CADD (Production) —> real time replicated (close to real time) to Public Safety Warehouse

• Scheduled task, 3 mins
• Stages in separate database - Oracle Server (RackWeb)
• CTM Downtown (RackWeb)

Question

• Scraping data from CTM, active traffic incidents
• ETL didn’t run, bug in code
  • gaps for traffic incidents

Data for downstream

• Data Warehouse
• Projects going for that

Historical Data

• Oleg is talking about what they’re trying to do
• Building a business case
• towards sharing data for customer
• customers with customers

Purpose - Reason for dataset

• Active incidents - traffic incidents currently

CADD Agency SPOCs

• Are there any problem natures for this?
• EMS isn’t sharing?
• Update some of these, some of these don’t exist

Updates from Fire

• wouldn’t see any changes

Summary

CTECC group suggests we don't get in the business of taking on ownership of data manipulation to make the data look better as the data belongs to other organizations. We can provide context around what the data is and isn't.

@KaroEngstrom and I plan to review the Draft page that Charlie has and create a data story about the data visualization we plan to have.