Lead QC process on ATD's APWA Practices during Citywide review

OPM organized a Citywide review of all APWA Practices being submitted in 2023. Using a series of excel spreadsheets, assigned reviewers (including Kate and Peggy) went through each and every Practice to check if it's up to the standards we need for full accreditation.

Now it's time for ATD to make revisions based on the QC review feedback. The fundamental steps are:

[x] Add columns for this phase to ATD APWA Progress Tracker
[x] Review feedback gathered during Citywide QC and copy/paste into Progress Tracker - Kate
[x] Communicate feedback to Leads - Kate
[x] Make revisions based on QC feedback - Leads
[x] Compile list of all evidence to see what's used several times (manuals, codes, etc.)
[x] Compile list of all outdated evidence that needs a revision or memo
[x] Final formatting following OPM guidelines - Kate
[x] Update all outdated evidence - coordinate among Leads, SMEs, & executives

Deadline for all revisions: May 17th Deadline for binder printing: June 1

Important note regarding the merger: The City of Austin formally applied to APWA for reaccreditation in January 2023 (before the merger) and will use its organizational structure from that date for the evaluation. We are not submitting documentation as TPW, but rather ATD, PWD, etc.

APWA QC Review Wrap Up & Next Steps

Meeting led by Stephanie F. on 4-14-23

New Timeline

May 17: Revisions due to OPM
May 31: Final documentation deadline; binders will be printed
May 31 at 1-3 PM: In-person training for evaluations led by Terry O. (certified evaluator) including live demo
June 13-14: Mock evaluations for partner departments (PWD will be earlier)
Late June: Dress rehearsal including PWD and all partner departments
June 30: Deadline to upload documentation to APWA site
July 10-12: In-person accreditation

Action Items

Complete QC revisions for all ATD Practices by May 17
Submit final documentation to be printed for APWA binder by May 31
Identify Chapter presenters
Submit survey to OPM re: evaluation
Help prepare presenter's binder

QC Review Feedback

The feedback gathered during review was color coded red, yellow, or green to help visualize work that needs to be done. Every department had a mix of red, yellow, and green in their feedback; that's the nature of the QC process.

Make It Easy for Evaluators

Compliance statement should be concise and prove the blue.
Only include necessary evidence to prove the blue.
- Minimize the number of examples included in your Evidence.
- Put the strongest piece of Evidence first.
- Highlight relevant sections in the Evidence documents that prove the blue.
Specify department/area of responsibility: The City of Austin's Transportation Department...
Ensure all documentation is updated per the formatting guidelines (numbering, labeling, etc.).

Procedures, Policies, Programs, & Memos

If the Practice asks for a "procedure in place," it must be:
- Official (not drafted)
- Current (after July 2019)
- Signed by authoritative figure
If we reference Policies, they must be:
- Official (not drafted)
- Current; if it's July 2019 or before, we must also provide an official memo stating the policy is still applicable today (OPM created a template for this)
- Example: City, State, and Federal ordinances and codes
- Example: We follow the State policy on xyz
If a Practice does not have an official Procedure or Policy, whatever we do is referred to as a Program:
- Summarized in a one-page document "COA's Transportation Department follows the processes provided in the flow chart"
- OPM created a template for this

Update ATD APWA Progress Tracker

Add columns for:

Citywide QC Review Status - choice column Peer review in progress Communicate revisions Revisions in progress Revisions complete

Practice Sheet Feedback - long text Copy/paste feedback emailed by Stephanie F.

Evidence Feedback - long text Copy/paste feedback emailed by Stephanie F.

Evidence Up To Date - yes/no If any piece of Evidence is dated July 2019 or before, it is not current and must be updated according to APWA.

List Outdated Evidence - long text List procedures, policies, code, ordinances, etc. that are dated July 2019 or before

Final Formatting by Kate - choice Waiting on revisions to be done Ready for final formatting Final formatting complete

Feedback on Chapter 16 - ROW Management

16.01 – Permit Issuance – Paloma, Ryan

[x] Ensure that evidence is listed in order of importance - the strongest file is first and the weakest is last.
[x] Is the first file all that is really needed to prove the blue statement?
[ ] 2-4 - Highlight specific sections that pertain to permit process. If there isn’t anything, delete that piece of evidence.
[x] 4 - Not current per APWA so we need a revision or memo stating it’s still valid.
[x] Align nomenclature & formatting

16.02 – Permitting Responsibility – Paloma, Ryan

[x] Ensure that evidence is listed in order of importance - the strongest file is first and the weakest is last.
[x] Are the first two files all that is really needed to prove the blue statement?
[x] 1-5 - Highlight specific sections that pertain to designating responsibility for developing and implementing permit procedures or controlling activity within the public rights-of-way. If there isn’t anything, delete this piece of evidence.
[ ] 1 - Not current per APWA so we need a revision or memo stating it’s still valid.
[x] Align nomenclature & formatting

16.03 – Codes and Permit Information – Paloma, Ryan

[x] Great work!
[x] Align nomenclature & formatting

16.04 – Permit Process – Paloma, Ryan

[x] Provide an example of how tracking is performed throughout a permit cycle in order to prove the “track the work to be done” part of the blue statement. This could be similar to Evidence 16.02.5, which shows the internal database screenshot.
[x] Align nomenclature & formatting

16.05 – Permit Inspection – Darren

[x] 1-2 - Highlight specific sections that pertain to a policy for inspection of permitted work. If there isn’t anything, delete this piece of evidence.
[ ] 1-2 - Not current per APWA so we need a revision or memo stating it’s still valid.
[x] Align nomenclature & formatting

16.06 – One-Call Center Coordination – N/A 16.07 – Utility Marking Guidelines and Color Code – N/A

[x] Ensure the compliance statement fully explains WHY this practice is not applicable to the City of Austin / our agency.
[x] Stephanie plans to ask APWA if we need examples for those that are not applicable.
[x] Align nomenclature & formatting

16.08 - Pavement Cut Restoration - Paloma [practice sheet] [evidence folder]

[ ] Needs to be completed (currently blank because we thought it was someone else's responsibility but turns out it's ours)
[ ] Align nomenclature & formatting

Feedback on Chapter 34 - Traffic Operations

34.01 - Traffic Policy and Procedure Manual - Cody, Eric

[x] Ensure that evidence is listed in order of importance so the strongest file is first and the weakest is last. Should the TMUTCD be first?
[x] 1 & 3 - How do these pertain to a developed traffic control policy and procedure manual? Highlight specific sections. If there isn’t anything and it doesn’t pertain, delete this piece of evidence.
[x] 2 - Not current per APWA so we need to confirm it's still valid (then we'll work with OPM & execs on a memo)
[x] Align nomenclature & formatting

34.02 - Traffic Control Devices - Cody, Eric

[x] Ensure that evidence is listed in order of importance so the strongest file is first and the weakest is last.
[x] 1, 2, & 3 - Highlight specific sections that pertain to traffic control devices. If there isn’t anything, delete this piece of evidence.
[x] 3 - Not current per APWA so we need to confirm it's still valid (then we'll work with OPM & execs on a memo)
[x] Align nomenclature & formatting

34.03 - Requesting Installation of Traffic Control Devices - Cody, Eric

[x] Ensure that evidence is listed in order of importance so the strongest file is first and the weakest is last.
[x] 4 - Update with a formal procedure that is signed and dated. This appears to be an unfinished draft with some text crossed out.
[x] Provide evidence that shows a screenshot of what the service request for traffic devices looks like when submitted or filled in the request service tab. You can also show a screenshot of what the service request looks like to the user once it has been submitted and how its progress is tracked in the system.
[x] Align nomenclature & formatting

34.04 - Traffic Signs Maintenance - Lauren, Cathy

[x] Ensure that evidence is listed in order of importance so the strongest file is first and the weakest is last.
[ ] 5, 6 & 7 - Not current per APWA so we need to confirm it's still valid (then we'll work with OPM & execs on a memo).
[x] Align nomenclature & formatting

34.05 - Statutory Speed Limits on Roadways - Cody, Eric

[x] Ensure that evidence is listed in order of importance so the strongest file is first and the weakest is last.
[x] 2, 3 – Only include specific sections that pertain to an adopted ordinance that specifies the statutory speed limit on residential roadways and on other streets where no signs are posted. (96 pages for 3 is excessive.)
[x] 2 – Not current per APWA so we need to confirm it's still valid (then we'll work with OPM & execs on a memo).
[x] 4 – Limit the number of examples to one or two. (70 pages is too much.)
[x] Align nomenclature & formatting

Practice 34.06 - Street Name Signs - Cody, Eric

[x] Ensure that evidence is listed in order of importance so the strongest file is first and the weakest is last.
[x] 2 – Not current per APWA so we need to confirm it's still valid (then we'll work with OPM & execs on a memo).
[x] Align nomenclature & formatting

Feedback on Chapter 35 - Parking

List here.

Communicating feedback to Leads

Schedule 30-minute call with Lead; invitation template below
Email with specific feedback; email template below
Call to go through specific feedback by Practice, deadline/schedule, and tease/mention evaluator training, mock evaluation, and accreditation evaluation (use email as an agenda)
Follow-up email for any outstanding items; template below

Meeting invitation template

Hello [NAME]! I hope you're doing well.

We once again need your help on the APWA accreditation project. This time around should hopefully be quicker than in the past. I need to communicate the feedback we received from the citywide QC review of APWA Practice [NUMBER], which is the purpose of this call. We'll also talk about the timeline and upcoming events since the in-person accreditation evaluation is quickly approaching (July 10-12).

I look forward to hopefully talking with you soon!

Kate

Email template

Subject line: Updates needed by Fri 5/12 to APWA Practice

Hello [NAME]!

The department once again needs your help on the APWA accreditation project. This time around should hopefully be much quicker 🤞

We received our feedback from the citywide QC review. First off, you did an excellent job filling out the practice sheet and saving documentation as evidence. Especially since we started from scratch—seriously, job well done! 👏

Here is the feedback for your Practice [NUMBER] – [TITLE]:

Ensure that evidence is listed in order of importance on the practice sheet so the strongest file is first and the weakest is last. This is to make it as easy as possible for the evaluator to understand that we meet the requirement. If the first piece of evidence is good enough, he/she may not even go to the next piece.
For evidence [#], highlight specific sections that pertain to [BLUE STATEMENT]. If there isn’t anything, delete this piece of evidence from the practice sheet and move the file to the Additional Documentation folder in the evidence folder.
Evidence [#] is not considered current and up to date according to APWA. Can you please confirm its contents are still valid and applicable today? Then we will work with our executive team produce a signed memo stating it has been reviewed and is still applicable today. This will fulfill APWA’s dating requirements, ensuring everything is current and dated August 2019 or after.

An important note regarding the department merger: The City formally applied for reaccreditation in January (before the merger) and will use the organizational structure from that date for the evaluation. That means we’re still submitting documentation as ATD this year and no changes need to be made in that regard. (For reaccreditation in 2027 and beyond, we will submit as TPW.)

The deadline to complete the list above is [DAY, DATE]. I know that’s a super quick turnaround and it’s already a busy time. Can you get it done or will you need some help?

The department is counting on each of us to pull through this last leg of the accreditation journey. And once we pass in July, there will be a celebration!

Please let me know if you have any questions. I’m here to help.

Thank you! Kate

Follow-up email template

Subject line: APWA QC review updates - last questions

Hey [NAME],

I'm working to close the loop for this round of APWA edits and wanted to check with you on a few things:

[PRACTICE NUMBER] - [TITLE]

Outstanding item 1
Outstanding item 2

Please reply to this email when these updates are complete.

Don't hesitate to reach out if you have any questions.

Thank you! Kate

EXAMPLE OF OUTSTANDING ITEMS

16.1 – Permit Issuance

Has the practice sheet been checked to ensure the evidence is listed in order of importance (so that the strongest file is first and the weakest is last)?
Can we highlight specific sections in these files that pertain to a permit process for construction activity and/or encroachment in the ROW:
- Evidence 2 - TCM Section 13
- Evidence 3 - Utilities Criteria Manual Sections 3, 4, & 5
- Evidence 4 - CoA SOP 99-01

16.2 – Permitting Responsibility

There’s one last SOP that needs an updated date of August 2019 or after: SOP-DAPCZ.pdf

16.8 – Pavement Cut Restoration

This practice sheet still needs to be filled out and documentation needs to be saved in the evidence folder. This practice fell through the cracks early on and it hasn’t been completed by anyone yet. Peggy messaged Paloma about it last week and we’re hoping to get it done ASAP. Please let me know if you need assistance.

This is finished since we moved onto the mock evaluation

cityofaustin / atd-data-tech