Standardize 3-1-1 metric % Completed On Time

[katelunceford]

TPW should implement a standard formula to calculate the 311 CSR metric "Percent Completed On Time," which is used by many different groups for various types of work. Currently this metric is calculated differently across different reports or dashboards, resulting in mismatching numbers for the same metric. Mismatching numbers cause confusion and distrust in what's being reported. We need to uncover the differences in reporting formulas and decide on one standard formula moving forward.

For example, SMD's dashboard reports [%] while the 311 SR Response dashboard reports [%]

• SMD % - in the “late” category, it includes any CSRs that are open at the time of the report. But some are simply in progress and they have longer service agreements than one month and/or it was submitted recently. They're not technically late but they're being counted as such.
  • This approach gives incentive to close SRs as soon as possible, but it doesn't paint an accurate picture of the work
  • Maybe we should include "in progress" to capture the work that's not on time or late yet
• Dash % - counts CSRs as late once they're actually late
• There is something in 311's system that flags SRs as overdue/late because there's a report sent out on that

To Do

• [x] Gather all 311 reports floating around TPW (TPW, legacy ATD, legacy PWD) #17024
  • To be able to answer: Who is sending what to whom and how often?
  • Carrie at 311 emailed a copy of all the 311 reports sent to TPW; reports are saved here
  • Meeting with 311 to go over the reports they send and different ways this metric is calculated
• [x] Fill in this spreadsheet to track variations of the metric #17025
  • Each row is group that uses this metric
  • Columns track the dashboard/report, data source, formula calculation, and notes
  • DTS will fill in Dash, SMD, & AMD
  • OPM will fill in SBO & Parking
• [x] OPM & DTS will check in again to look at all the different ways we are calculating the measure
  • To decide if we're ready to bring this forward to leadership or if more work is needed
• [x] Present our findings and recommendations to the executive team to make decisions #17026
  • Summarize info gathered in a presentation format
  • Schedule time with Jim D.
  • Present to Jim D. with a goal to identify one formula TPW uses moving forward
• [ ] Update formula across all reports & dashboards to the new standard #18173
  • Coordinate with SMD so they're aware of the change & ask that they update their dashboard
  • Coordinate with any other report recipients/owners who will be affected
  • Coordinate with 3-1-1 to update reports

[katelunceford]

Notes from Data Science Retro on 4/16/24

• There are SO MANY people who use/enter info on CSRs at TPW - and a lot of them do it differently
• Trying to fix this in Maximo seems like a big technical challenge
• DTS can’t fix this because its dependent on business processes & DTS doesn’t have the power to change that - maybe we could partner with OPM
  • OPM can work on the business processes
    • Define what completed on time means for each work group
    • Increase the service level agreement visibility
    • Write a report on all the ways the dept uses CSRs + a standardization recommendation that would better track and show the work being done across the department
  • DTS can open up the data to make these problems visible
    • Who uses what report / data owners
    • Formula to calculate % Completed on Time

[ChristinaTremel]

Meeting with OPM on 4/23

• OPM is happy to help us navigate the conversation of standardizing the 311 Completed on Time reporting
• OPM will be reaching out to Parking and SBO because these two use the 311 CSRs completed on time as a performance measure and will find out which report they use to capture that metric
  • We will then review that report to determine how the measure is calculated
• Christina created a shared document and will include documentation on what report Arterial Management and Signs & Markings are using to calculate this metric
• After that, OPM and DTS will check in again and look at all the different way we are calculating the measure.
• Potentially will want to bring our findings and options for calculating the the executive team so they are able to participate in the decision making

[katelunceford]

Notes from Data Science Backlog Refinement on 4/24/24

• Carrie at 311 said we can change reports
• In the past, 311 has said that you can close a CSR if the work is scheduled
• If we report on CSRs and their status via Maximo or Knack and there's a problem with one of those systems, then there's a problem with our data. It would be better to use data directly from the source - 311

[katelunceford]

Notes from OPM Sprint Review on 4/30/24

• This metric will be part of TPW's KPIs so it's important we work it out & good we're tackling it now
• We anticipate the City will ask TPW for more reporting on KPIs & goals for FY25 and beyond