Document recommendations for City's evluation RFQs

[theecrit]

Description

The City of Oakland has an RFQ (request for qualifications) out for the next cycle of Measure Z evaluators. If we can get the city to build in certain data sharing protocols as part of the RFQ for vendors, it could make it easier for us to get more consistent data in the right structure, without having to reverse engineer from PDFs. But we need to know what data we’d want and in what form.

Examples of places in the RFQ that reference metrics

The DVP will work with the selected contractor to determine the best research design and metrics to use in evaluating each of the DVP’s strategies. Not all programs can be evaluated in terms of recidivism, but if this metric is chosen for some portion of the evaluation, please note that the City prefers the use of the Chief Probation Officers of California (CPOC) definition, which can be found in Section V (Definitions) of this RFQ. Additionally, the City prefers for an evaluator to use a Results-Based Accountability (RBA) structure, if possible. The RBA definition is also in Section V (Definitions).

Evaluation deliverables should be usable and accessible by the public, government stakeholders, and funded program providers. Deliverables will include: • Annual process evaluation reports for each of the DVP’s 11 sub-strategies (anticipated Oct 2023 and 2024)…

…Specific activities of the evaluation partner will include: • Development of all aspects of evaluation design, including the selection of metrics and the development of data collection tools and protocols, in collaboration with the DVP.

Running list of potential recommendations/requests

Each item should include the request and why it would be beneficial to the public to do so.

• Provide full data sets for any data used in deliverable reports, in CSV or similar format.
• Provide a copy of raw data that backs any graph or chart that gets presented in a report -- as well as the raw narratives that get summarized in more 'qualitative' sections.
• Use the same categories/definitions for data across strategies or agencies to allow more consistence mapping and tracking over time.
• Your idea here...

To close this issue, the following requirements should be met

Check off as you go to track progress. This list can be changed if needed.

• [ ] Gather a list of requirements/requests above.
• [ ] Review, clean and organize the list with the FPS team (with criteria for doing so documented in the comments below or alongside the list itself).
• [ ] Consolidate the list into a shareable document.

[jaedoak]

I wonder if they want to select some categories/definitions/protocols from the last reports that must also be used in the next reports, so that trends can be reasonably evaluated over time?

[theecrit]

Welll I just received word that they've selected their evaluators for the next cycle, and are about to announce them. I'm seeing if we might get an introduction to build a relationship there.