Standardize 3-1-1 metric % Completed On Time

[katelunceford]

TPW should implement a standard formula to calculate the 311 CSR metric "Percent Completed On Time," which is used by many different groups for various types of work. Currently this metric is calculated differently across different reports or dashboards, resulting in mismatching numbers for the same metric. Mismatching numbers cause confusion and distrust in what's being reported. We need to uncover the differences in reporting formulas and decide on one standard formula moving forward.

For example, the calculated percentages on the SMD dashboard and the TPW 3-1-1 Service Request Response dashboard do not match.

• SMD % - in the “late” category, it includes any CSRs that are open at the time of the report. But some are simply in progress and they have longer service agreements than one month and/or it was submitted recently. They're not technically late but they're being counted as such.
  • This approach gives incentive to close SRs as soon as possible, but it doesn't paint an accurate picture of the work
  • Maybe we should include "in progress" to capture the work that's not on time or late yet
• Dash % - counts CSRs as late once they're actually late
• There is something in 311's system that flags SRs as overdue/late because there's a report sent out on that

To Do

• [x] Gather all 3-1-1 generated reports floating around TPW (TPW, legacy ATD, legacy PWD) #17024

• [x] Fill in this spreadsheet to track variations of % Completed On Time #17025

• [x] OPM & DTS review the different ways we are calculating the measure, decide we need leadership's input

• [x] Present findings to Jim D. with a goal to identify one formula moving forward #17026

• [ ] Update SMD's formula across all reports & dashboards to the new standard #18173

• [ ] Identify the recipients, frequency, & purpose of each 3-1-1-generated report & if we could use Power BI instead #18348

• [ ] Show report recipients how to use the TPW 3-1-1 dashboard

• [ ] Coordinate with 3-1-1 to update/stop reports

[katelunceford]

Notes from Data Science Retro on 4/16/24

• There are SO MANY people who use/enter info on CSRs at TPW - and a lot of them do it differently
• Trying to fix this in Maximo seems like a big technical challenge
• DTS can’t fix this because its dependent on business processes & DTS doesn’t have the power to change that - maybe we could partner with OPM
  • OPM can work on the business processes
    • Define what completed on time means for each work group
    • Increase the service level agreement visibility
    • Write a report on all the ways the dept uses CSRs + a standardization recommendation that would better track and show the work being done across the department
  • DTS can open up the data to make these problems visible
    • Who uses what report / data owners
    • Formula to calculate % Completed on Time

[ChristinaTremel]

Meeting with OPM on 4/23

• OPM is happy to help us navigate the conversation of standardizing the 311 Completed on Time reporting
• OPM will be reaching out to Parking and SBO because these two use the 311 CSRs completed on time as a performance measure and will find out which report they use to capture that metric
  • We will then review that report to determine how the measure is calculated
• Christina created a shared document and will include documentation on what report Arterial Management and Signs & Markings are using to calculate this metric
• After that, OPM and DTS will check in again and look at all the different way we are calculating the measure.
• Potentially will want to bring our findings and options for calculating the the executive team so they are able to participate in the decision making

[katelunceford]

Notes from Data Science Backlog Refinement on 4/24/24

• Carrie at 311 said we can change reports
• In the past, 311 has said that you can close a CSR if the work is scheduled
• If we report on CSRs and their status via Maximo or Knack and there's a problem with one of those systems, then there's a problem with our data. It would be better to use data directly from the source - 311

[katelunceford]

Notes from OPM Sprint Review on 4/30/24

• This metric will be part of TPW's KPIs so it's important we work it out & good we're tackling it now
• We anticipate the City will ask TPW for more reporting on KPIs & goals for FY25 and beyond