Foundation Mission Request - Evaluating Voting Design Tradeoffs for Retro Funding

To take on this project, submit a proposal to this thread by June 7th. Read more about Missions here.

Foundation Mission Request Summary: Evaluate tradeoffs of different voting designs for Retroactive Public Goods Funding ("Retro Funding") based on simulations and present conclusions in a table.
S5 Intent: Intent 4: Improve governance accessibility
Proposal Tier: Ember
Baseline grant amount: 9,000 OP
Should this Foundation Mission Request be fulfilled by one or multiple Alliances: Multiple
Optimism Foundation point-of-contact: @elizarileyoak until Jun 10, @JSeiferth after Jun 10
Submit by: June 7
Selection by: June 14

How will this Foundation Mission Request help accomplish the above Intent?:

In order to reach our goal of improving governance accessibility, we need to understand the optimal voting design to encourage diverse, informed, and values-aligned voting behavior.

To recap, we refer to governance accessibility as follows:

“Accessibility” includes enabling a diversity of perspectives to participate in governance, facilitating better knowledge sharing to develop more informed voters, and lowering barriers to participation for more culturally diverse involvement in the governance process. Increasing the votable supply and reducing the concentration of voting power are important byproducts of improved accessibility.”

What is required to execute this Foundation Mission Request?

This Foundation Mission aims to measure how different Retro Funding voting designs perform against a number of requirements aimed at optimizing different objectives. We hope to identify a voting design which minimizes the impact of "malicious badgeholders" on voting outcomes, while also achieving our other requirements (e.g. incentive compatibility, easy to understand for voters, etc.). The performance of different voting designs against requirements should be achieved by simulating different types of voter behavior and applying formal reasoning.

Requirements to test voting designs against

For each of the below requirements you should apply a standardized way of measurement across voting designs

Resistance to malicious behavior: Measure how a malicious voter can impact voting outcomes
Resistance to collusion: Measure how a group of coordinated malicious voters can impact voting outcomes
Incentive compatibility: Prove or disprove that the voting design is incentive compatible (”every participant can achieve their own best outcome by acting according to their true preferences”)
Simplicity for voters: Voters can easily understand how the voting design works and how to best behave to achieve their goals
[Your input here! We’d love to hear from you about requirements we should optimize for]

Voting designs to test

In Retroactive Public Goods Funding, voters are asked to express how much OP a project should receive based on the impact they delivered to the Collective.

Quadratic Voting: Retro round 1 leveraged quadratic voting
Mean: Retro round 2 applied the mean to calculate results
Median: Retro round 3 applied the median to minimize the impact of outliers
[Your input here! We’d love to hear from you what voting designs should be tested]

Resources:

The desired outcome of this mission is:

A memo making a recommendation based on both qualitative and quantitative research:

The primary output should be a table comparing different voting designs (e.g., quadratic voting, median voting) against several high-priority requirements that align with Collective goals (e.g., COI-resistance, incentive compatibility, ease of voting process)
The table contents should be based on data simulations of voter behavior to gain insights on how different voting designs perform. This analysis should be shared in an appendix or in a separately linked repo, and underlying assumptions in each model should be stated explicitly
There should also be a written section summarizing the table, noting the tradeoffs between different voting designs with respect to each requirement

What milestones will help the Collective track progress towards completion of this Foundation Mission Request?

1. Identify list of requirements and voting designs to evaluate

Before proceeding to the analysis, the Foundation and the grantee have agreed on the final requirements that will be optimized for and the voting designs that will be tested based on a proposal by the grantee
This should include a brief summary of each requirement and voting design to justify its importance, juxtapose tradeoffs, and list assumptions

2. Quantitatively test voting designs via data simulations

Grantee has developed a simulation framework to test different voting designs
In the analysis, it is most helpful to assume the base case of voting for a Retro Funding project where a voter has a set amount of OP tokens to allocate across a given number of projects
- The model likely includes design parameters such as number of votes, minimum allocation, and quorum requirements

3. Compile findings and recommendations via written memo

The grantee has delivered a memo which includes a table outlining tradeoffs across these dimensions and a written portion evaluating tradeoffs, as described above

How should badgeholders measure impact upon completion of this Foundation Mission Request?

The summary of tradeoffs between these voting designs with respect to the different requirement should be clear, both quantitatively in the table and qualitatively in the written portion.
If this Mission Request is successful, Collective stakeholders will be able to evaluate different voting designs through the lens of a few prioritized requirements.
Ideally, the mission directly informs the voting design for upcoming retro funding rounds. In this case, grantees are encouraged to apply for an additional allocation of tokens in the August 2024 governance-themed Retro Funding Round 6.

Application instructions

To apply for this RFP, please complete the form in the expandable section below and leave your response as a comment on this issue thread. Submissions will be open until June 7, at which time the Foundation will review all submissions and select one or two individuals/teams to complete the work defined here.

Submission form

_Copy the entire application below and leave a comment on this issue with your answers completed. A representative from the Optimism Foundation may reach out using the contact info provided to request more information as necessary._ ## Foundation Mission (RFP) Application **Please verify that you meet the qualifications for submitting at the above [Tier](https://gov.optimism.io/t/collective-trust-tiers/5877/2)** * **Alliance Lead:** Please specify the best point of contact for your team * **Contact info:** * **L2 recipient address:** * **Please list the members of your Alliance and link to any previous work:** Read more about Alliances [here](https://gov.optimism.io/t/season-4-alliance-guide/5873) --- **What makes your Alliance best-suited to execute this Mission?** - [...] - [...] **Please describe your proposed solution based on the above Solution Criteria (if applicable):** - [...] - [...] **Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each peice of work:** - [...] - [...] **Please define the [critical milestone(s)](https://gov.optimism.io/t/grant-policies/5833) that should be used to determine whether you’ve executed on this proposal:** - [...] - [...] **Please list any additional support your team would require to execute this mission (financial, technical, etc.):** - [...] - [...] **Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant:** _(Note: there is no guarantee that approved Missions will receive up-front cash grants.)_ - [...] --- Please check the following to make sure you understand the terms of the Optimism Foundation RFP program: - [ ] I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance. - [ ] I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant - [ ] I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the [Operating Manual](https://github.com/ethereum-optimism/OPerating-manual/blob/main/manual.md#valid-proposal-types) - [ ] I confirm that I have read and understand the [grant policies](https://gov.optimism.io/t/token-house-grant-policies/5833) - [ ] I understand that I will be expected to following the public grant reporting requirements outlined [here](https://gov.optimism.io/t/suggested-public-reporting-requirements-for-grantees/4176) -- end of application --

Alliance Lead

Simer Singh

Contact info

Email: simers228@gmail.com
Username: @simer_14

L2 recipient address

0xE61b17c7601E1FC839bD59c6313e7f92A4926eBB

Alliance members and previous work

GitHub
Most recent project: A visualization presenting a side-by-side comparison of a standard Ethereum transaction and a transaction that leverages Zero-Knowledge Proofs (ZKPs)

What makes your Alliance best-suited to execute this Mission?

With a strong foundation in Quantitative Economics and extensive hands-on experience in data engineering and analytics, I am well-equipped to tackle this mission. My background in developing AI-driven tools and my contributions to the Ethereum Foundation have provided me with the necessary skills to create robust, scalable data systems and run complex simulations. This project perfectly aligns with my technical expertise in Python and deep interest in the field of blockchain governance.

Proposed solution

My proposal involves developing a Python-based simulation framework to evaluate the performance of existing voting mechanisms (Quadratic, Mean, and Median Voting) and introduce an AI-enhanced double-blind voting system. This approach randomizes project assignments to voters and initially masks project details using AI-generated descriptions, reducing bias and voter fatigue. By focusing on core aspects and withholding identifying details, the system encourages merit-based assessments and minimizes collusion. Once voters approve their assigned projects, they can allocate their OP and access more detailed information. This solution addresses key requirements, including minimizing collusion, aligning incentives, increasing informed voting, and reducing voter fatigue. Crucially, this approach tackles the subjectivity of equating impact to profit by assigning projects to voters based on their expertise and interest, ensuring a more accurate and context-aware assessment of each project's value. The simulation framework will provide insights into the performance of various voting mechanisms, informing future retro funding round designs.

Step-by-step plan and deadlines

June 2024: Develop a Python-based simulation framework to evaluate Quadratic, Mean, and Median Voting mechanisms. This step lays the groundwork for more complex simulations.
July 2024: Integrate an AI-enhanced double-blind voting system into the framework. This includes both the randomization of project assignments and generating AI-created project descriptions to ensure unbiased evaluations.
August 2024: Run a series of comprehensive simulations to compare the performance of all voting mechanisms. The analysis will focus on meeting the specified requirements, followed by compiling the findings into a detailed report.

Critical milestones

Completion of the Python-based simulation framework that supports evaluating different voting mechanisms.
Fully operational AI-enhanced double-blind voting system within the framework, proving the system's capability to manage randomized assignments and generate project descriptions.
Successful execution of detailed simulations and the subsequent creation of a memo that compiles the results along with clear recommendations based on the analysis.

Additional support required

Collaboration with the Optimism Governance team for insights and feedback throughout the project.

Terms of the Optimism Foundation RFP program

[x] I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance.
[x] I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant.
[x] I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the Operating Manual.
[x] I confirm that I have read and understand the grant policies.
[x] I understand that I will be expected to follow the public grant reporting requirements outlined here.

Alliance Lead: Luuk Weber (LuukDAO)
Contact info: TG: LuukDAO / email: luuk@kolektivo.co
L2 recipient address: 0xfAD384709D78fDd8B3Cb557a9943bfaab0a31391

Alliance members and previous work:

LuukDAO: Lead of CeloPG and BD Superchain Eco - 5+ years experience designing and operating onchain organizations and programs. Created programs, including Balancer Grants and CeloRPGF0, that allocated over USD 5m onchain.
Jashdotfi: Lead of Superchain Eco - 3+ years experience as Web3 / Finance analyst. Holds an MSc degree in Financial Economics.
Sodofi: Former PGN and Active CeloPG Steward - 2+ years experience coordinating onchain public goods ecosystems.
MontyMerlin: Co-founder ReFiDAO and Active CeloPG Steward - Designed and operated 5+ onchain grant programs in collaboration with Gitcoin, CCN, and Hypercerts.

What makes your Alliance best-suited to execute this Mission?

We bring a unique combination of theoretical knowledge, practical experience in onchain program and voting design, and a deep understanding of the OP Collective.
As active participants in all layers of onchain voting and resource allocation systems, we have the right context and incentives to progress the understanding of vote design efficiency for the OP Collective.

Proposed solution

Our Alliance will leverage existing data from RPGF 1/2/3, CeloRPGF0, GG20, and the Citizen, Project, and Chains database of Superchain Eco to identify patterns to shape our assumptions and models about the various voting designs.

We will run a quantitative (Python) and qualitative simulation (using role-playing) to gain insights into each voting design's effectiveness based on this FMR's requirements.

The results will be presented in a Mirror blog post and Memo, including an easily digestible table breaking down each voting design's performance and ability to allocate resources effectively per OP Collective objectives.

Requirements to test voting designs against In addition to the four requirements specified in the FMR, we will also analyze 5. The applicant reporting workload, and 6. The degree to which the voting process can be automated.

Voting designs to test In addition to the three designs specified in the FMR, we will also analyze 4. Conviction Voting, and 5. Incentivized Conviction Voting (where reviewers can earn upside for identifying impact).

Step-by-step plan and deadlines

Based on our experience and given that this group has already coordinated successfully, we estimate the project to take 6 weeks to complete.

Phase 1 | Retro analysis and modeling | Week 1-2: The team will analyze existing sources, including RPGF 1/2/3, CeloRPGF0, GG20, and the Citizen, Project, and Chains database of Superchain Eco, to shape the assumptions and define the simulation phase.

Phase 2 | Simulation | Week 3-4: The team will create different test cases leveraging quantitative methods (Python) and qualitative methods (role-playing) to gain insights into the effectiveness and tradeoffs of each voting design.

Phase 3 | Synthesis, Conclusions, and Recommendations | Week 5-6: The findings from the Simulation phase will be further analyzed and used to create conclusions about the effectiveness and tradeoffs of each voting design, which will be presented in a table and written format.

Critical milestones

Critical Milestone 1: Deliver a document describing the research and results from the Retro analysis phase and how these insights have shaped the simulations at the end of Phase 1, approximately 2 weeks after starting the FMR. Critical Milestone 2: Deliver a document describing the simulation progress, including raw data at the end of Phase 2, approximately 4 weeks after starting the FMR. Critical Milestone 3: Publish a Mirror Blog and share a Memo, including an easily digestible table and write-up of the tradeoffs and scores of each vote design at the end of Phase 3, approximately 6 weeks after starting the FMR.

Additional support required

If available, get access to any additional data/information around RetroPGF 1/2/3 that can be helpful in our research.

Terms of the Optimism Foundation RFP program
[x] I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance.
[x] I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant.
[x] I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the Operating Manual.
[x] I confirm that I have read and understand the grant policies.
[x] I understand that I will be expected to follow the public grant reporting requirements outlined here.

Foundation Mission (RFP) Application

Please verify that you meet the qualifications for submitting at the above Tier

Alliance Lead: Jay Yu, President of Stanford Blockchain Club
Contact info: jyu01@stanford.edu
L2 recipient address: oeth:0xD5b5e91B416004C5503929377445B30d964cf090
Please list the members of your Alliance and link to any previous work:

Research Team: Jay Yu ‘25, Austin Bennett ‘25, Brian Grenadier ‘22 Law ‘25, Billy Gao ‘27, Rebecca Joseph ‘27 – Stanford University Faculty Advisor: Prof. Jeff Strnad, Charles A. Beardsley Professor of Law, Stanford Law School

What makes your Alliance best-suited to execute this Mission?

All members of our research team have either been students, researchers, or teaching staff of Stanford University’s “CS 352B/LAW 1078: Blockchain Governance”, a class dedicated to legal and structural concerns of blockchain governance.
Our members have represented Stanford Blockchain Club as Uniswap Delegates, and will serve as Optimism Token House Delegates for Season 6
Multiple members will write their graduating senior thesis on the topic of DAOs and decentralization and possess prior publication experience, including an in-depth analysis of the implementation of Quadratic Voting.
Our alliance lead is working with Prof. Ari Juels at Cornell Tech on applying the Voting Bloc Entropy metric to Optimism’s Citizen House
Our members have organized the Stanford Blockchain Governance Summit (SBGS) in April 2024 alongside the CodeX group at Stanford Law School
Our faculty advisor, Prof. Jeff Strnad, has recently authored and presented “Economic DAO Governance: A Contestable Control Approach” at multiple conferences including the TLDR Conference in May 2024. This paper represents a paradigm shift in DAO governance.

Please describe your proposed solution based on the above Solution Criteria (if applicable):

Our goal is to produce a Systemization of Knowledge (SOK) of various voting design patterns, assessing the impact of adversarial behavior and collusion on each of these different designs.
The voting designs we currently plan to test include:
1. Quadratic Voting, based on Retro round 1 design
2. Mean Voting, based on Retro round 2 design
3. Median Voting, based on Retro round 3 design
4. Generalized Non-quadratic Polynomial Voting - generalized results to other forms of polynomial voting schemes
We will evaluate against all metrics listed in the RFP above (resistance to malicious behavior, collusion, incentive compatibility, and simplicity for voters) . Additionally, we intend to examine adversarial behavior by modeling non-voters as Jensen-Meckling free-riders for the DAO
We will base our results from mathematical and theoretical derivations, simulation-based techniques, and natural experiments based on existing Optimism voting history.
We aim to publicly publish our work on Stanford Blockchain Club’s magazine Stanford Blockchain Review and release our simulation tool-kit in an open-source format.
In the future, we may work with Optimism to implement more experiments of novel proposed voting schemes (e.g. general polynomial voting mechanisms) as a team or as a representative of Stanford Blockchain Club

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each peice of work:

Step 1: Literature review and formulate precise mathematical definitions for target voting techniques and evaluation criteria. Step 2: Create mathematical proofs and derivations of expected theoretical results. Step 3: Create code-based simulations of these techniques to instantiate baseline. Step 4: Run natural experiments on Optimism’s historical data, evaluate differences with theoretical results. Step 5: Formulate into written form and publish results, conclusions, and areas for further research.

Please define the critical milestone(s) that should be used to determine whether you’ve executed on this proposal:

June-July: Complete Step 1 July-August: Complete Steps 2 & 3 August-September: Complete Step 4 October: Complete Step 5

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

Context and support on the implementation and results of previous rounds of RPGF, and qualitative conversations with Optimism stakeholders in forming hypotheses.
Technical APIs and data support on previous voting infrastructure

Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant: (Note: there is no guarantee that approved Missions will receive up-front cash grants.)

We would like to have a cash grant of around 2000-3000 USD to fund initial operations and upfront costs, in either OP or stablecoins.

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

[x] I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance.
[x] I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant
[x] I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the Operating Manual
[x] I confirm that I have read and understand the grant policies
[x] I understand that I will be expected to following the public grant reporting requirements outlined here

-- end of application --

Foundation Mission (RFP) Application

Please verify that you meet the qualifications for submitting at the above Tier

Alliance Lead: David Zhou, Cofounder at Doxamarket & Governance Co-lead at Blockchain@USC
Contact info: lczhoudaily@gmail.com
L2 recipient address: 0x5c1D15dC3da43A18E60C2DdB87B3466097855530

Please list the members of your Alliance and link to any previous work:

David Zhou | BA Economics @ UC Berkeley & MS Analytics @ USC:
- RetroPGF S3 recipient for Onchain Governance Metrics Dashboard
- ETH Denver 2024 DAO-track Finalist
- Incentive Analysis NumbaNerd Bounty on Yearn and Velodrome
Ziyue Wei | MS Analytics at USC:
- Data Analyst at Tesla
- MLOps Research Assistant at USC

What makes your Alliance best-suited to execute this Mission?

Blockchain@USC has been a dedicated and impactful delegate in the Optimism ecosystem for nearly two years, consistently earning retro delegate rewards through active governance participation. Alliance member Liangchen Zhou has contributed public governance dashboards monitoring onchain governance metrics of a16z Optimism delegates and general delegates at Optimism and developing award-winning governance tools to address key issues in the DeGov space. Additionally, the University of Southern California boasts exceptional data talent, exemplified by Ziyue Wei's Machine Learning research under Professor Wilcox and her industry experience at Tesla. Combining our domain expertise with seasoned industry data professionals, our alliance is uniquely positioned to execute governance-domain research with precision and efficiency. We are confident in our ability to complete this grant successfully.

Please describe your proposed solution based on the above Solution Criteria (if applicable):

Determine Metrics Requirements
- Resistance to malicious behavior: Measure how a malicious voter can impact voting outcomes
- Measure: $PercentageChange = \frac{(\text{Score with Malicious Voters} - \text{Score without Malicious Voters})}{\text{Score without Malicious Voter}} * 100$
  - Average percentage change across all projects.
  - Maximum percentage change for any single project.
- Resistance to collusion: Measure how a group of coordinated malicious voters can impact voting outcomes
- Measure: $PercentageChange = \frac{(\text{Score with Collusion Voters} - \text{Score without Collusion Voters})}{\text{Score without Collusion Voter}} * 100$
  - Average percentage change across all projects.
  - Maximum percentage change for any single project.
- Incentive compatibility: Prove or disprove that the voting design is incentive compatible (”every badgeholders can achieve their own best outcome by acting according to their true preferences”)
- Measure: Consensus Proximity SSR
  - Lower values of average SSR indicate that the final scores are very close to the consensus scores, suggesting high incentive compatibility. Higher values indicate discrepancies between what voters collectively believe and what the system outputs, suggesting potential issues with incentive compatibility.
- Simplicity for voters: Voters can easily understand how the voting design works and how to best behave to achieve their goals
- Assess subjectively
- Computational Scalability: voting system can still work well when handling an increasing number of voters and projects
- Measure: Average processing time
- If the average processing time increases linearly or sub-linearly with the number of voters and projects, the system demonstrates good computational scalability.
Determine Voting Design
- Quadratic Voting: Sum of square root
- Mean: Simple mean
- Median: Simple median
- Trimmed Mean/Median: Remove 25% of the highest and lowest scores before calculating to reduce the impact of outliers, then calculate the mean/median
- Multiple Rounds of Sampling Mean/Median: Conduct 10 rounds of random 10% sampling, the mean or median is calculated for each round, and then the average of these values is taken
- Dirichlet Process: Votes are clustered using a Dirichlet Process Gaussian Mixture Model (DPGMM). The means of these clusters are then averaged to produce the final score
Develop Simulation Framework
- build on this OSO simulator
- Assumptions: voting design follows the Metric-based Evaluation
- Impact Vectors
  - Community Engagement Metrics
  - V1: Total Stars
  - V2: Total Forks
  - V3: Contributors Last 6 Months
  - V4: Avg Monthly Active Devs Last 6 Months
  - User Adoption and Activity Metrics
  - V5: Total Onchain Users
  - V6: Onchain Users Last 6 Months
  - V7: Total Txns
  - Financial Metrics
  - V8: Total Txn Fees (ETH)
  - V9: Txn Fees Last 6 Months (ETH)
  - Vector values are standardized, range from 0-1
  - Vector values are right skewed: more projects have lower vector values
  - Voters can randomly select 1-9 vectors according to their preference and measure of impact
  - Voters can assign total 100 weight to vectors they selected
- Voter Category
  - HonestVoter: Select 3-9 vectors
  - MaliciousVoter: Single Metric Focus: Create a subset of voters who disproportionately allocate their votes to a single metric or a small set of metrics, ignoring others.
  - CollusionVoter: Identical Voting Patterns: Generate groups of voters with identical or highly similar voting patterns, indicating coordinated behavior.
- Class
- Project
- Voter
- Vote
- Round
- Simulation
- Test
Quantitative testing of different requirements to assess the performance of voting systems
Compile findings and make recommendations

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each peice of work:

[x] Milestone 1: Identify list of requirements and voting designs to evaluate
Steps
- Before proceeding to the analysis, the Foundation and the grantee have agreed on the final requirements that will be optimized for and the voting designs that will be tested based on a proposal by the grantee
- This should include a brief summary of each requirement and voting design to justify its importance, juxtapose tradeoffs, and list assumptions
Estimated Dates: June 7th, 2024

Milestone 2: Quantitatively test voting designs via data simulations

Steps
- Grantee has developed a simulation framework to test different voting designs
- In the analysis, it is most helpful to assume the base case of voting for a Retro Funding project where a voter has a set amount of OP tokens to allocate across a given number of projects
- The model likely includes design parameters such as number of votes, minimum allocation, and quorum requirements
Estimated Hours:
- Simulation: 2 weeks
- Quantitative Testing: 1 week

Milestone 3: Compile findings and recommendations via written memo

Steps
- The grantee has delivered a memo which includes a table outlining trade-offs across these dimensions and a written portion evaluating tradeoffs, as described above
Estimated Dates:
- 3 days

Please define the critical milestone(s) that should be used to determine whether you’ve executed on this proposal:

Proposal complete (doc): Identify list of requirements and voting designs to evaluate
Simulator implementation (Python script): According to requirements to develop the voting simulator based on existing class
Quantitative testing (Jupyter notebook): Using the simulator to complete the analysis, making adjustments to the simulator as needed
Memo writing: Table of dimensions evaluation, outlining trade-offs

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

Compute credits if needed

Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant: (Note: there is no guarantee that approved Missions will receive up-front cash grants.)

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

[x] I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance.
[x] I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant
[x] I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the Operating Manual
[x] I confirm that I have read and understand the grant policies
[x] I understand that I will be expected to following the public grant reporting requirements outlined here

-- end of application --

Foundation Mission (RFP) Application

Please verify that you meet the qualifications for submitting at the above Tier: ✅ Alliance Lead: Angela Kreitenweis Contact info: angela@tokenengineering.net L2 recipient address: 0xA449d4f0DB455ea661CDF9fA09F670382aC6FB76

Please list the members of your Alliance and link to any previous work:

Angela Kreitenweis – Angela has been active in token engineering and DAO governance design since 2018. With TE Academy she has launched an educational program on token-based voting and governance. She is building open-source LLM tools for system and governance engineering and is active in the crypto community to drive educated, data-based decentralized decision-making. With TE Academy’s educational program, she has successfully participated in RetroPGF rounds 2 and 3.

Nimrod Talmon – Nimrod is a researcher and a professor with a focus on Social Choice Theory and published >70 scientific papers in the domain, e.g. multiwinner elections, liquid democracy, and participatory budgeting. Collaborating with crypto projects like DAOStack and TE Academy, he has broad experience in developing, analyzing, and implementing voting systems in crypto. He is also teaching undergraduate and graduate courses on the foundations of blockchain, DAO governance, social choice, and Monte Carlo simulations.

Muhammad Idrees – Muhammad is an AI/ML Engineer with a background in Robotics. He brings a wealth of expertise in system modeling, agent-based simulations, decentralized economic systems modeling, and integrating Large Language Models (LLMs) with decentralized economic systems.

Eyal Briman – Eyal is a PhD candidate and a researcher in social choice, with broad experience in agent-based modeling and simulations. He holds an MSc in Data Science, graduating magna cum laude. His research focuses on Application-Oriented Social Choice with Structural Constraints for both the input and output of social choice procedures.

What makes your Alliance best-suited to execute this Mission?

Proven Track Record Our team members have an extensive and proven track record in relevant fields such as DAO governance design, modeling, and simulations. We have been collaborating since 2020, bringing together a unique combination of scientific research, applied modeling and simulations (including AI/ML/LLM applications), and expertise in governance accessibility and education.

Alignment with Mission We have been active participants in the crypto DAO space for several years, contributing to communities like Gitcoin, DAOStack, TEC (Token Engineering Commons), and Optimism. Our shared curiosity for research and experimentation, and our commitment to discovering coordination mechanisms and sustainable value flows, align perfectly with this mission. As proud recipients of RetroPGF with TE Academy, we are dedicated contributors to the ecosystem, offering education and platforms for discussion and research.

Strategic Value Add Our vision is to enhance data-based, informed decision-making in DAOs while supporting a diverse range of perspectives within the decision-makers’ collective. By combining our skills, we create synergies between voting design, governance accessibility, and artificial intelligence. We are actively pursuing this vision by engaging in this evaluation project, developing innovative LLM tools and frameworks, running experiments on token-based voting, and conducting governance research.

Please describe your proposed solution based on the above Solution Criteria (if applicable):

Optimism Retro Funding is one of the most visionary endeavors in the crypto space. Building a sustainable system that measures impact and distributes rewards accordingly is huge. The current voting design has successfully facilitated funding for a diverse range of projects, evaluated from an equally diverse set of badgeholder perspectives. Still, it has to evolve further. With Retro Funding 4’s Metrics-based Evaluation, Optimism is entering the next stage of Retro Funding.

We’ll evaluate the voting design tradeoffs for Retro Funding to complement these efforts.

We’ll explore: Resistance to malicious behavior: We consider different malicious behaviors and the possible effect they have on the outcome – this includes malicious actions such as strategic voting (non-honest voting to change the outcome favorably, also referred to as manipulation); control (adding dummy projects); and bribery (affecting the way other badgeholders behave).

Resistance to collusion: We consider different forms of adversarial collusions and their effect on the outcome – this includes group manipulation (in which groups of badgeholders vote strategically to change the outcome, possibly with internal-group monetary compensations; this relates to game-theoretic concepts such as coalition strategy-proofness and the core).

Incentive compatibility: We evaluate whether every participant can achieve their own best interests by acting according to their true preferences.

Robustness of the rule: Evaluating the possible effect that a badgeholder – or a group of badgeholders – has on the final outcome.

Representativeness of the rule: We evaluate the representativeness of the results – how they reflect the preferences of groups of badgeholders, including the effect that minority groups have on the outcome and are being represented by it.

Diversity of the results: We evaluate the diversity of the results, and how the selected projects correspond to the votes of different kinds of badgeholders.

Diverse, informed, and values-aligned voting behavior: We measure the representativeness of the votes and diversity of results (majority vote, minority vote, population fractions).

Simplicity for voters: We evaluate how intuitively the voting mechanism can be understood by voters, where appropriate we suggest tooling to support the voting process and maximize accessibility.

We apply the following evaluation methods:

Mathematical analysis: Many of the criteria can be formulated as binary predicates, meaning that a specific voting design either satisfies the criteria or not (e.g., whether strategic voting exists or not) – for these criteria, we will mathematically analyze their satisfaction. Furthermore, other criteria can be formulated in a way that admits theoretical analysis, e.g., what is the maximum fraction of the budget that can be affected by strategic voting – for these, we will analyze such bounds theoretically.

Agent-Based Modeling: We will develop agent-based models to simulate the behavior of individual voters and groups of voters. This will include parameters such as badgeholder voting preferences, project/metrics variability, and strategic behavior of badgeholders. This will be useful for evaluating, e.g., the effect of different kinds of strategic behavior on the outcome for different scenarios. We’ll cover different parameters of quorum thresholds, votes, and minimum allocation in our agent-based model, and sweep these parameters for voting design configuration, which is resistant to malicious behaviors and collusion. To account for randomness and variability in voter behavior, we implement Monte Carlo simulations. This will involve running hundreds of thousands of simulation iterations to observe different voting scenarios and outcomes.

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each piece of work:

1. Requirements and voting designs to evaluate

1.1 Research & data analysis

Optimism Retro Funding 360°: past rounds voting design, infrastructure, stakeholders, issues, potential attack vectors; future rounds requirements and constraints
Historical Data Analysis: Historical voting data from previous RetroPGF rounds to calibrate and validate the simulation models; this ensures the models accurately reflect real-world behavior

1.2 Voting Design to evaluate: Looking at the requirements, we select the voting designs to be verified. We’ll cover:

voting designs in prior rounds Optimism Retro Funding rounds (Quadratic, Mean, Median)
variants of these designs, such as adapted quotas and quorums required, and respective parameter settings.
new voting designs to address the requirements – viewed as variants of divisible participatory budgeting or portioning – specifically the two state-of-the-art solutions, namely Moving Phantom mechanisms (in particular, the independent markets mechanism) and Nash Maximizer.

1.3 Priorities: Agree on final requirements with stakeholders, prioritize to derive the evaluation plan, and kick off the simulation phase.

2. Quantitatively test voting designs via data simulations

2.1 Evaluation Plan & Simulation Framework: For all requirements, we’ll propose a standardized way to measure performance across voting designs. In the evaluation plan, we’ll define a standardized way of measurement across voting designs. As a base case, we’ll use Retro Funding’s current ballot design where a voter has a set amount of OP tokens to allocate across a given number of projects.

2.2 Running Simulations: There’s a range of approaches to verifying a voting design in simulations:

Resistance to Malicious Behavior

Scenario Simulation: Simulate scenarios where a single malicious voter attempts to manipulate the voting outcome. Measure the impact of their actions on the final allocation of funds.
Metric: Calculate the deviation from an ideal, fair allocation under different voting designs. Resistance to Collusion

Group Collusion Modeling: Simulate scenarios with groups of coordinated malicious voters. Test how different group sizes and strategies affect the voting outcome.

Metric: Assess the extent to which collusion can alter the allocation, using metrics such as the Gini coefficient and vote entropy. Incentive Compatibility

Behavioral Simulation: Model voter behavior under the assumption of rational utility maximization. Simulate different voting strategies and observe outcomes.

Metric: Measure the Nash equilibrium to determine if the voting design is incentive-compatible, ensuring that truthful voting is the dominant strategy.

Data Collection and Analysis

Data Logging: Log all simulation results, including voting patterns, allocation outcomes, and performance metrics.
Comparative Analysis: Compare the performance of different voting designs using statistical methods. Visualize the results in charts and tables for clarity.

In the evaluation plan, we’ll specify the methods we apply and select the approaches that fit the requirements best.

3. Compile findings and recommendations in a written memo

3.1 Report Table with quantitative results comparing different voting designs against high-priority requirements that align with Collective goals Qualitative report: Summary of results, noting the tradeoffs between different voting designs with respect to each requirement Appendix: simulation results in detail, including metrics, experimental setup, parameter settings, and underlying assumptions to be aware of. Public repository: we’ll make the framework of our simulations available open-source

3.2 Recommendations Recommendations on best voting design Badgeholder guide for evaluating different voting designs through the lens of prioritized requirements What-if scenarios exploring future rounds to inform the voting design for upcoming retro funding rounds

Beyond phases:

This project is poised to create synergies with our ongoing experiments in applying AI agents to DAO voting and decision-making. We are eager to explore how Large Language Models (LLMs) can help make our results more accessible and how AI agents can be integrated into the badgeholder voting tool stack for the recommended voting design. With demonstrated value added, we plan to participate in Retro Funding Round 6 (Governance) in September.

Please define the critical milestone(s) that should be used to determine whether you’ve executed on this proposal:

Milestone 1 – Mid July Requirements doc agreed upon and submitted (see 1. Requirements and voting designs to evaluate) Milestone 2 – Mid August Quantitative Tests completed (see 2. Quantitatively test voting designs via data simulations) Milestone 3 – End of August Report and Recommendations delivered (see 3. Compile findings and recommendations in a written memo)

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

For exploring past Retro Funding round performance and issues, we’ll need access to raw, anonymized voting data.

Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant: (Note: there is no guarantee that approved Missions will receive up-front cash grants.)

Not a barrier.

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

✅ I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance. ✅ I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant ✅ I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the Operating Manual ✅ I confirm that I have read and understand the grant policies ✅ I understand that I will be expected to following the public grant reporting requirements outlined here

-- end of application --

Foundation Mission (RFP) Application

Please verify that you meet the qualifications for submitting at the above Tier

Alliance Lead:

Jade Clyne

Contact info:

jade.clyne@proton.me

L2 recipient address:

0x690ab933C5082049d20E53B21Dd37F9A346C8B47

Please list the members of your Alliance and link to any previous work:

Jade Clyne: Jade is the Programs Lead of Token Engineering Commons (TEC). She has an extensive background in analytics - from working on Andrew Yang’s political campaign to working in nonprofits building equity. Projects include: tracking demographics, economic indicators, conducting & analyzing community needs surveys, conducting federal treasury reports with the Southern Economic Advancement Project (SEAP). Participating in Census Open Innovation Labs (COIL) sprint helping communities access infrastructure grant funding. As well as applying Tunable Quadratic Funding on TEC's latest grant round and writing simulation tests to determine the best weights for voting
YGG Anderson: With over a decade of experience in data science and software engineering, the founder of Longtail Financial has led the company to explore the intersection of exponential technologies in finance, focusing on machine learning, decentralized coordination, and token engineering. He has contributed to Block Science, Bonding Curves Research Group (BCRG), and TEC on TQF. Previously he contributed to impactful projects such as developing AI-driven financial models for sustainable development at First Green Bank Network and reducing access management overhead at RBC Capital Markets by over 90%. His technical expertise includes Python, TensorFlow, Django, and deep learning, with a proven track record in creating scalable, innovative solutions that drive ecological and community well-being.
Rex: An economist and data analyst with over half a decade in experience in varied crypto economic and governance systems. Rex has contributed to novel data visualization and simulation models at the TEC, BCRG and Wonderland.
Octopus: A Ph.D. mathematician with 15 years experience using programming to discover the properties of systems. Octopus has contributed to more than two dozen cryptocurrency projects involving detailed simulations, data visualization, risk analysis, and attack design. Some of the teams 8 have worked with include Token Engineering Academy, GitCoin (Fraud Defense and Detection workstream), FYDE Treasury Protocol, BlockScience, and TokenDynamics.
Tamara Helenius: The co-founder of Commons Stack and advisor to the Token Engineering Commons (TEC), having previously served as steward since 2020. Most intrigued by mechanism designs and web3 potential in this regard, esp as applied to governance and economics. Prior to crypto rabbit-holing in 2016, she was the global head of delivery for a BBDO agency in Paris, managing consultant at Capgemini, and led digital programs at TheStreet.com, Google and Sony Music in New York.
Gideon Rosenblatt: Gideon has been a steward and advisor to the TEC for the past two years. He is a long-time social entrepreneur with a focus on mission-driven technology. He led product development teams for large-scale, consumer-facing web services at Microsoft. He also ran a technology consulting firm for the environmental movement, where he learned much about grant making and impact measurement.
Bear: Bear has been part of the TEC for more than two years, fulfilling various roles. Currently, serving as the Coordination Lead, supporting initiatives, managing team operations, and shaping strategic directions. His expertise spans organizational development, innovation, and social entrepreneurship. Passionate about DAOs, governance, and exploring innovative ways for humans to work and collaborate together.
Nate Suits: Nate has been with the TEC for the last four years participating in many roles, serving as a Steward and is currently the Events Lead. He holds MPA/MPP degrees, has worked for several political campaigns, and serves as a part-time community organizer in Oakland, California focusing on improving low-income housing services. He is interested in mechanism design, governance, and new methods for public funding.

What makes your Alliance best-suited to execute this Mission?

Data Expertise + Community Grant Making

This alliance represents a partnership between data experts and grant round operators. The data team has extensive experience in data science, big data, machine learning, and overall analytics, and has been actively involved in developing the techniques proposed here for much of this last year. This capacity will be important in informing the specification and evaluation of the different voting designs outlined below and in rigorously testing and simulating the various options for these designs. The grant round operators have hands-on experience with the ins and outs of running community-determined capital allocation rounds. They have run four Gitcoin Quadratic Funding (QF) rounds on behalf of the Token Engineering Commons (TEC), distributing over $175,000 in funds.

Part of the Optimism Ecosystem

This team is closely affiliated with the TEC, a DAO that is in the process of embedding itself within the Optimism ecosystem. We are honored to be past recipients of OP Retro PGF funding. In December of last year, the TEC migrated our DAO from Gnosis to OP Mainnet. We were also the only prominent grant round that chose to run on OP Mainnet in the most recent Gitcoin Grants (GG20). We are deeply invested in Optimism and see this project as another way we can contribute to the ecosystem’s success.

Field-Tested Proposal

Over the course of four Gitcoin QF rounds, we have carefully implemented, tested, and revised the variable-driven vote weighting techniques proposed here. The first implementations of this ‘vote tuning’ technique focused on QF as a voting mechanism, hence the name, “Tunable Quadratic Funding” (TQF). It is a proven technique that our data experts continue to evolve and that our grant operators have now field tested four times with excellent results. This combination of data expertise and hands-on work in using it to manage Gitcoin rounds with thousands of voters makes this team well-suited to working with Optimism to research new voting designs for Retro Funding.

Please describe your proposed solution based on the above Solution Criteria (if applicable):

Solution

Our proposal entails analyzing voting designs based on two categories of design elements: statistical measures of “tuning variables” and evaluation of requirements. Tuning variables are generally framed in terms of specific values that matter to a given community. In the case of the TEC, we use vote tuning to combine QF’s strength in democratic processes with signals Subject Matter Expertise (SME) and other values that matter to our community. Tunable QF does this by looking for specific tokens and attestations associated with the wallets of voters and boosting their voting signal when we find them. For example, we use the $TEC token and “proof of knowledge” NFTs from the Token Engineering Academy as proxies for token engineering SME, and use that to boost the votes of those donors.
For this project, we propose generating a matrix of design elements that combine statistical measures such as median, mean, etc. and combine these with various tuning variables (see below). We will then run simulations on these combinations with the goal of identifying voting designs optimized to encourage diverse, informed, and values-aligned voting behavior.

Tuning Variables

We have already tested a variety of different types of signals as proxies for tuning variables, including tokens, NFTs, and EAS attestations. When a specific signal, such as a particular type of attestation, is detected in the wallets of voters, that vote is ‘boosted’ by a certain percentage. Signals can be combined so that a particular type of NFT might boost a vote by 2X and a particular attestation by an incremental 1.5X.
Tuning variables represent a safe process for testing various approaches to increasing the inclusiveness, accessibility, and scalability of OP Retro voting. We assume all votes will continue to be associated with badge holders, though we can simulate future designs that treat badge holder attestations as the highest-boosted among several other tuning variables. It will be important to collaborate with the Foundation to select the most useful tuning variables, but here are a few suggestions:
1. Badge Holder Signals: Weight votes based on badge holders’ level of past voting engagement
2. Delegate Signals: Weight votes from OP delegates
3. Token Holder Signals: Weight votes based on various thresholds of OP token holding (this could be weighted low but could one day be used to greatly expand Retro Funding voting accessibility without undermining the importance of the badge holder signals)
4. OP-selected Signals: We see an opportunity to collaborate with the Optimism team to select a handful of other signals to incorporate into this test, perhaps from the various Optimism Citizen Attestations.

Statistical Measures

We plan to combine the following statistical measures with tuning variables to evaluate and optimize different voting designs:
1. Mean
  - Purpose: To understand the central tendency of voting results
  - Application: Calculate the average vote weight for each tuning variable to see how they influence the overall voting outcome
2. Median
  - Purpose: To identify the middle value in a distribution of votes, reducing impact of outliers
  - Application: Use the median to assess the central tendency of votes without being skewed by extreme values, providing a more robust measure of typical voting behavior
3. Standard Deviation and Variance:
  - Purpose: To measure the dispersion or variability of vote weights
  - Application: Evaluate how widely votes are spread out from the mean
4. Herfindahl-Hirschman Index (HHI)
  - Purpose: To assess the concentration of voting power
  - Application: Calculate HHI to determine if voting power is overly concentrated among a few participants or more evenly distributed
5. Gini Coefficient
  - Purpose: To measure inequality in vote distribution
  - Application: Use the Gini coefficient to assess the fairness of vote distribution
6. Entropy
  - Purpose: Measure the randomness or diversity of vote distributions
  - Application: Calculate entropy to evaluate the diversity of voting patterns
7. Correlation Coefficients
  - Purpose: To assess the relationship between different tuning variables and voting outcomes
  - Application: Use Pearson or Spearman correlation coefficients to identify how strongly tuning variables are associated with vote weights and outcomes
8. ANOVA (Analysis of Variance)
  - Purpose: To compare the means of voting outcomes across different groups or tuning variable levels
  - Application: Conduct ANOVA tests to determine if there are statistically significant differences in voting results based on various tuning variables

Requirements:

To meet the requirements for testing voting designs, we will develop a comprehensive python simulation framework. This framework will incorporate the following tests to evaluate each voting design based on:
1. Resistance to Malicious Behavior: We will simulate scenarios where a malicious voter attempts to manipulate voting outcomes. By introducing controlled variables that allow a single voter’s impact to be exaggerated, we will measure the extent to which each voting design can resist this behavior.
  - Hypothesis: Each voting design will resist manipulation by a single malicious voter, maintaining outcome integrity despite attempts to exert disproportionate influence.
  - Methodology: Measure the change in voting outcome when a single voter is given exaggerated influence. Incorporate Passport Score attestations in our tests. Create scenarios where a single voter’s vote weight is increased significantly. Compare the final outcomes with and without malicious influence across different voting designs, while maintaining consistency in the number of voters.
2. Resistance to Collusion: Our Simulations will model groups of coordinated malicious voters. We will analyze how these groups can affect voting outcomes under each design and measure the resilience of each system.
  - Hypothesis: Voting designs will be resilient to coordinated manipulation by groups of voters, minimizing the impact of collusion on the final outcomes.
  - Methodology: Measure the extent of outcome deviation when groups of voters collude. Simulate voting scenarios with varying sizes and strategies of colluding groups. Model coordinated voting behavior among groups of voters. Assess the deviation in outcomes and ability of each design to minimize the impact of collusion. Vary the size and strategy of colluding groups while keeping other parameters constant. Test the designs under different levels of voter coordination and strength of collusion.
3. Incentive Compatibility: We will test whether each voting design is incentive compatible. This means evaluating if participants can achieve their best outcomes by voting according to their true preferences. Our simulations will include different voter profiles to ensure that the designs promote honest voting behavior.
  - Hypothesis: Each voting design will be incentive compatible, promoting honest voting behavior as the most beneficial strategy for participants.
  - Methodology: Measure the alignment between reported (simulated) and true preferences and the outcomes for honest vs strategic voting. Note that this is an area that needs to be developed in collaboration with the foundation.
4. Simplicity for Voters: We will assess how easily voters can understand each voting design and how straightforward it is for them to participate effectively. This will involve feedback from a diverse group of voters to measure the ease of understanding and participation.
  - Hypothesis: Voters will find each design easy to understand and participate in, resulting in higher usability scores and lower error rates.
  - Methodology: Conduct surveys/interviews, collecting qualitative and quantitative feedback on experiences of those who participated in voting designs being tested. Use a set of heuristics or principles to evaluate each voting design (e.g. clarity, consistency, error prevention), generating a heuristic score based on how well it adheres to the principles.
5. Accessibility: We will measure accessibility by evaluating how well each voting design enables diverse participation and lowers barriers to involvement. Our framework will include:
  - Hypothesis: Voting power distribution and participation metrics will reflect fairness and inclusivity in each voting design
  - Methodology: Ensuring voting power is not overly concentrated and participation is broad and representative. Tracking number of participants, voting power distribution, concentration indices, inclusivity of signals. We can use concentration indices like the Herfindahl-Hirschman Index (HHI) to assess voting power distribution. Create visualizations to illustrate participation and power distribution

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each piece of work:

Identify List of Requirements and Voting Designs to Evaluate
- Month: June
- Deliverables:
  - Collaborate with the Foundation to finalize the list of requirements and voting designs (including tuning variables)
  - Summarize each requirement and voting design, highlighting its importance and tradeoffs
  - List assumptions for each design and requirement
- Target Metrics:
  - Finalized list of requirements and voting designs
  - Summary document of each requirement and design
Develop and Implement Simulation Framework
- Month: July
- Deliverables:
  - Develop a Python simulation framework to test different voting designs
  - Simulate various scenarios, including edge cases for malicious behavior, collusion, and incentive compatibility
  - Conduct feedback surveys of OP Retro Funding voters in the past to measure simplicity
- Target Metrics:
  - Complete a functional simulation framework
  - Comprehensive simulation data for each voting design
Compile Findings and Recommendations via Memo or Forum Post
- Month: August
- Deliverables:
  - Create a memo with a comparative table of statistical measures with tuning variables, and outlining trade offs across dimensions such as resistance to malicious behavior, collusion, incentive compatibility, and simplicity
  - Provide a written evaluation of tradeoffs, noting qualitative and quantitative insights
  - Share an appendix or link to a repository containing detailed simulation data and underlying assumptions
- Target Metrics:
  - Memo including comparative table of voting designs
  - Written section evaluating trade offs
  - Detailed simulation data shared

Please define the critical milestone(s) that should be used to determine whether you’ve executed on this proposal:

Milestone 1: Finalization of Requirements and Voting Designs
- Achieved by end of June
- Successful collaboration with the Foundation to finalize the requirements and designs
Milestone 2: Development and Completion of Simulation Framework
- Achieved by end of July-mid August
- Creation and testing of the simulation framework for all voting designs
- Gather feedback from past OP Retro Funding voters
Milestone 3: Compilation and Submission of Findings
- Achieved by end of August
- Delivery of the memo and detailed analysis to the Optimism Foundation

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

Access to raw voting data of Retro Funding rounds
Support for organizing community engagement events and promoting participation and transparency throughout the testing process to gather diverse feedback on voting designs

Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant: (Note: there is no guarantee that approved Missions will receive up-front cash grants.)

No barrier for us.

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

[x] I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance.
[x] I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant
[x] I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the Operating Manual
[x] I confirm that I have read and understand the grant policies
[x] I understand that I will be expected to following the public grant reporting requirements outlined here

Foundation Mission (RFP) Application

Alliance Lead: Kiran Karra
Contact info: kiran@cel.build
L2 recipient address: 0x6c23BD747Bbe603582525d7C77d5C690AC461da3
Please list the members of your Alliance and link to any previous work:

Kiran, Tom and Team from CryptoEconLab.

Examples of previous work:

What makes your Alliance best-suited to execute this Mission?

Our team has used agent-based simulation to design cryptoeconomic incentives for several protocols. Our team intends to solve this problem by expanding OpenSourceObserver’s agent-based simulator. We have confidence in this approach having already adapted the OSO code to set voting parameters (quorum, scoring rule etc) for the Optimism-inspired FIL-RetroPGF-1 round.

Other experience from the team in creating and communicating agent-based modeling include:

An agent-based simulation of network resilience and published analysis of L1 network response under a wide range of multitype agent behaviors.
Modeling demand for gas usage for an L1 VM using agent based modeling which informed the design space for gas pricing, as well as publishing open sourced analysis.
Creation of differentiable mechanistic model of an L1 cryptoeconomy, which was used to learn optimal parameters for achieving optimization goals.
Developing web-based tools to communicate in a simple way the output of complex simulations, for example, accounting for the interaction between agents and an L1 protocol, and simply communicating how this affects rational miner choices:
- FIL Storage Provider Cost Explorer
- Proof of Data Possession Incentive Calculator

Our team’s modeling experience, combined with booting and operating a retro round, give us useful firsthand context, and we’re doubly aligned, as the output is highly relevant to inform the next FIL-RetroPGF-2 round too.

Please describe your proposed solution based on the above Solution Criteria (if applicable):

Approach

Our approach is to upgrade the existing simulation software to components that interact to simulate a Retro Funding round, create and run simulation configurations that are of interest, and analyze them for creating actionable outcomes that help inform future round designs in Optimism.

We identify three components that work together to create a simulated Retro Funding round design, and generate the output data to assess the effectiveness of the round design:

Agents—Model badgeholders and the various behaviors we want to test for. The current OP-RPGF-3 software simulates agents that consider some additional factors, such as badgeholder laziness and expertise, but it does not model negative behaviors, such as badgeholder collusion and negative strategic voting, that can corrupt the Retro Funding process. We will rearchitect the current simulator to include these additional agent behaviors while also allowing for future additions by community members.
Voting design - the method by which badgeholder preferences for projects are translated to votes and, eventually, funds distribution. OP and FIL RetroPGF rounds, to date, have used quorum-based voting designs. Here, badgeholders cast their available votes across all projects and certain thresholds. However, other voting designs are also valid for Retro funding. They should be tested quantitatively to determine if these designs are better for the outcomes that we desire (robustness to negative behaviors, understandability, etc ..). These include:
- Ranking + Distribution - In this voting design, we first define a mechanism by which projects are rank-ordered by badgeholders. Two immediate possibilities are a) direct ranking, where badgeholders, instead of casting votes, rank order projects according to the goals of the round (such as maximum impact within impact window), or b) ranks generated through pairwise comparisons (here, badgeholders are presented with pairs of projects and must compare the two, after which a process converts these pairwise rankings into a global ranking of all projects). The ranks are then translated to a funds distribution, which can be computed on a curve or defined a priori.
- Positive + Negative Signaling - In this voting design, badgeholders will be able to not only positively vote for a project, but also negatively. This is inspired by financial markets, which have mechanisms by which participants can signal positive (long) or negative (short) sentiments.
- Confidence Signaling - In this approach, badgeholders vote not only for the amount of funding they believe each project should receive but also provide an assessment of their confidence for each vote. This could serve to more accurately translate badgeholders desired funding distributions.
Funds Allocation - Each voting design in b) must have a protocol for translating the votes cast to funding distribution. The parameter space for translating votes cast to funding distribution is large, and will partially depend on the voting design chosen. However, it is possible that multiple voting designs can leverage multiple funds allocation mechanisms.

The three components identified will be integrated into an Agent-Based Model simulation. Since agents will have stochastic behavior, Monte Carlo simulations will be used to generate summary statistics of the effectiveness of a particular round design configuration.

We plan to generate simulation configurations of interest and run Monte-Carlo simulations for each configuration. A simulation configuration consists of:

The population distribution of the badgeholder - for example, 70% honest, 20% COI, and 10% colluding.
The voting design
The fund allocation mechanism.

The results of each simulation will be aggregated, and output metrics (defined below) measured and presented in a simple decision matrix table.

Software design note: Our approach is to build generalizable software components that can be extended by the community for future experimentation beyond the cases considered in this Mission. Practically, this means that new agent behaviors can be added to the software framework and leverage an existing voting design, or vice versa. This will enable future round designs to be discovered and experimented with rapidly.

Outputs For each monte-carlo run, the following metrics that assess the effectiveness of the round design will be computed:

A distance measure of the Project Impact vs. Allocation (since we are in a simulation environment, we can assign project impact and simulate badgeholder’s impact assessment), versus a desired Project Impact vs. Allocation.
Distribution of funds - is it exponentially distributed, etc?
Robustness of results to negative behaviors, as a function of the percentage of the population that behaves negatively, and conditioned on various combinations of negative behaviors.
- KPI: the breakdown point of a particular round design is the percentage of bad actors in the badgeholder population that causes the distance between the desired impact vs. allocation and the actual impact vs. allocation to exceed a threshold. This KPI can be a key quantitative metric for rank ordering round designs.

Additionally, we will seek to develop quantitative metrics that can assess the simplicity of the voting design. The following concepts will be explored:

The amount of explanation needed to describe, in plain language, the way that votes are translated into distribution. This can be translated into quantitative metrics, such as the number of words or characters needed to describe the method, as well as more algorithmic approaches such as Kolmogorov Complexity.
- A challenge identified here is to compress the description to be as parsimonious as possible, which is a subjective. We hypothesize that this can be overcome by standardizing the method by which the description is pruned, before complexity is computed - such as through the use of LLM’s and pre-defined prompts.
The number of corner cases that arise and need to be explained.
We will investigate ways in which the metrics above can be translated into a quantitative measure of simplicity, that can also be used to rank order voting designs.

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each piece of work:

Gather more fine-grained requirements from Mission funders (1-2 weeks).
Develop the software framework that simulates various agent populations, voting designs, and funding allocation mechanisms (1mo).
Create simulation configurations and run simulations to gather analysis data (1mo).
Analyze and collate results into document to be submitted to OP Mission (1mo).

Please define the critical milestone(s) that should be used to determine whether you’ve executed on this proposal:

Open source software with upgraded agents, scripts to run simulations, and Jupyter notebooks to produce figures that will be uploaded to GitHub as a pull request to the existing simulator by the end of the engagement (3.5 months from the start).
Final report laying out the results of the simulations, and recommendations for round configuration to make round resilient to negative behaviors while amplifying positive ones. This report will be uploaded to a publicly viewable location of the Mission’s choice (we have worked with Medium, HackMD, and GitHub) by the end of the engagement (3.5 months from the start).

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

We would like to work closely with the team in charge of running this mission to ensure that we fully understand the requirements and deliver an impactful report that can inform future round designs.

Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant: (Note: there is no guarantee that approved Missions will receive up-front cash grants.)

N/A

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

[X] I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance. [X] I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant [X] I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the Operating Manual [X] I confirm that I have read and understand the grant policies [X] I understand that I will be expected to following the public grant reporting requirements outlined here

Thanks to everyone for the excellent submissions ✨ The applications are now closed and we'll select teams to fulfil the mission by June 14th

The following teams have been selected for this Foundation mission:

Alliance lead: @jayyu23
Alliance lead: @AngelaKTE
Alliance lead: @kkarrancsu

If you've been selected, we will contact you via email.

To all the other teams that applied, we'd love to help you find the right way to contribute to the Optimism Collective! ✨ There are more Delegate Mission requests and Foundation Missions to following soon. In addition, there are builder ideas, which provide guidance on contributions which are valuable to the Collective and could be Retro Funded.

ethereum-optimism / ecosystem-contributions