Modification: Allocators and Meta-Pathway Design

[ghost]

Fil+ System Design: Allocators and Meta-Pathway Design

Issue Description

The Filecoin Plus (Fil+) program has been undergoing significant growth since its launch back in 2019. With each new phase, the program has created new pathways to accommodate clients' needs.

The program currently consists of four different pathways, GLIF, Client Onboarding, Large Dataset Notary (LDN), and Enterprise Fil+ (E-Fil+), showing signs of potential bottleneck and limitations in addressing evolving use cases. As the ecosystem has evolved, it has uncovered a myriad of new use cases which necessitate the introduction of fresh programs and pathways. For many users today, Fil+ is largely perceived as solely offering the LDN pathway for DataCap access, which may deter potential clients with new and unique use cases from participating in the program.

Despite this perception, Fil+ has never been a singular route but a versatile, meta-level program. The program can and should support the creation and operation of various “allocator” mechanisms for distributing DataCap.

Every Fil+ community member with interest, ideas and resources, should feel enabled to apply themselves to design and run their unique DataCap allocator mechanism if there is not one available to meet their needs.

This proposal for the next election cycle hopes to address these main issues:

• Change the narrative about Fil+ from being limited to the LDN pathway
• Reframe and broadcast the program as a meta-level program with an array of potential allocator pathways
• Introduce a clear process enabling community members to craft and propose their distinctive allocation pathways

Impact

Adaptability and Scalability: With the introduction of new pathways, the program showcases its adaptability. Not only does this evolution prepare the network to seamlessly handle a vast number of clients and diverse use cases, but it also emphasizes the commitment to scaling with future demands.

Efficiency and Responsiveness: Recognizing that a one-size-fits-all approach is often insufficient, the expanded pathways underlying Fil+'s dedication to delivering efficient, responsive, and tailored solutions. Clients from varied regions, industries, and objectives can now benefit from pathways that resonate with their specific needs. This client-centric approach ensures that every user feels valued, understood, and catered to.

Engagement and Participation: By continuously refining and expanding the program, Fil+ is set to increase engagement from clients and organizations worldwide. This proactive transition ensures that potential participants always find a pathway that aligns with their aspirations. Such an inclusive approach not only strengthens the Filecoin community but also positions Fil+ as a beacon for innovation and user engagement in the decentralized storage space.

Proposed Solution(s)

Introduce New Allocator Pathways: Entities/Organizations that can facilitate the distribution of DataCap within Fil+.

Application Process for Allocator Pathways: Entities interested in creating their own pathways should undergo an application process addressing key concerns:

How to create an Allocator pathway:

Step 1: Application Submission Platform: Aspiring Allocators should begin their journey by filling out the application in GItHub and ensuring all fields are thoroughly addressed. This application will live in the “Notary Governance” repository. The template will have sections for notary applicants to describe how they will distribute DataCap and ensure compliance with the program goals. For example:

• Who are the users?
• How will the Notary ensure data alignment with their pathway?
• What is the distribution methodology, and how will the Notary and Allocator pathway monitor deal distribution?
• What is the budget or funding plan to support this allocation pathway?
• What tooling will be used?
• How will the Notary bookkeep and handle disputes?
• How will the Notary iterate and adapt on the pathway over time? Are there specific change mechanisms that will be considered?

Step 2: Application Review Process Review Panel: Submitted applications will undergo a review process led by the governance team with additional support from some active notaries, Filecoin Foundation staff, and possibly other engaged Protocol Labs Network team members. If you would like to help review notary applications, please let us know! This ensures a holistic and unbiased evaluation.

Scoring System: Applications are scored based on the election rubric, reflecting the applicant’s capacity, intent, and alignment with the program objectives.

Step 3: Approval and Onboarding Notification: Successful applicants will be notified and announced to the community upon their selection as Allocators.

Diligence and Wallet Verification: Chosen notary allocators will need to complete some additional KYC diligence and tooling onboarding. This will include verification of the notary’s wallet.

DataCap Allocation: Approved and verified notary allocator pathways will be awarded the designated amount of DataCap that was approved in their application.

Note: The process of becoming an Allocator is inspired by the Notary election procedures of the past.

Client Application and Allocator Selections:

• Many clients will still apply for DataCap by accessing filplus.storage.
• As part of the original application process, clients will be prompted with a question asking them to select their preferred Allocator and corresponding pathway.
• Post application submission, the client's information will be automatically pushed to a dedicated GitHub repository, ensuring transparency and open-source diligence and bookkeeping.
• With the chosen pathway, the Allocator will be auto-assigned to this application on GitHub, enabling them to view and act on the request promptly.
• This workflow not only streamlines the DataCap request process but also ensures full transparency. Every stakeholder, including other clients and the broader Filecoin community, can view the client details, their application specifics, and the Allocator in charge.

Note: This process will not be required for all allocator pathways, and each allocator is responsible for designing and implementing their client-onboarding, diligence, bookkeeping, and on-chain message construction. The above pathway will be made available and maintained by the Fil+ working group.

Creation of New Allocator Pathways:

Establishing New Allocator Pathways: With the ecosystem’s evolution, the necessity to introduce diverse pathways has become evident. Some examples of what pathways could look like:

• Manual Diligence: Involving human intervention ensuring a meticulous inspection and onboarding the data that is aligned with the notaries goals. For example, the E-Fil+ and Open Datasets pathways.
• Market Based Allocator: Operating on different ideas such as auction houses, Trustless Notary, etc.
• Automated Allocator: Leveraging automated processes such as GLIF’s GitHub O-Auth.

Defining the Allocator Pathway Creation Process:

Custom Allocator Tooling & Workflow:

Allocators are empowered to employ their own tooling and workflows if our default flow doesn't meet their requirements. Allocators opting for a custom route should provide a detailed breakdown of:

• The tools they intend to employ.
• A step-by-step workflow of how clients' requests will be handled.
• Their backend management system specifics, detailing whether they will be using our default GitHub repository or have their own tailored solution.
• If Allocators wish to use our tools with theirs, such hybrid systems are welcome but need to be discussed in advance to ensure seamless integration.

New workflows and solutions to make the system almost platform agnostic are welcomed, however, before any custom tool or workflow gets adopted, it would be consulted upon to ensure it aligns with the overall objectives and standards of the program.

Timeline

Application Process Opens: Q4 2023

Application Review: Q4 2023 - Q1 2024

New Allocator Pathway Announcement: Q1 2024

Allocate DataCap to Notaries: Q1 2024

Technical Dependencies

Some tooling changes for the Fil+ registry app would need to be spec'ed and implemented.

Some tooling changes across various public dashboards would be required to more accurately track signatures for allocations and then client DataCap usage.

Risks and Mitigations

Risk of Centralization/Abuse: Allocators could become centralized hubs of DataCap distribution and only allocate to certain clients which introduces bias choice making.

• Mitigation: Implement a cap on how much DataCap a Notary can distribute and periodically review their distribution patterns. Success metrics will be needed to be in place to show the growth of these Notaries.

Misalignment with Fil+ Objectives: Notaries might prioritize their own commercial interests over the broader objectives of Fil+.

• Mitigation: Ensure clear guidelines for each Allocator pathway, with periodic audits and reviews.

Inefficient Distribution or Misuse: A Notary might not effectively distribute DataCap or might misuse it.

• Mitigation: Implement stringent monitoring mechanisms, possibly using automated tools, and provide clear consequences for misuse or inefficient distribution.

Increased Complexity for Clients: With multiple Allocator pathways, clients might find the system more complex to navigate.

• Mitigation: Create a unified platform or portal where clients can easily find and navigate through different Allocator pathways, perhaps supported by user-friendly guides and tutorials.

Conclusion

Overall, while problems may arise, the new system design will start off with a small amount of DataCap to each approved allocator pathway and the overall system can iterate accordingly over time. Please note that, two key drafts are in the pipeline:

• The election rubric
• An application template for prospective allocator pathways

Community members and notaries, your insights are needed. As the program transitions and expands, your feedback on these changes, especially on new pathways, is paramount. Watch out for the upcoming drafts.

[Asher774]

The crisis that fil+ is facing right now is clearly not caused by a lack of New Allocator Pathways, are you guys in the wrong direction?

[Chris00618]

Currently we all know there is too little real storage demand and too many allocator pathways that create ungovernable fraud. So this proposal sounds a little bit strange.

We should discuss how to incentivize the real clients to join filecoin instead of increasing more allocator pathways.

[herrehesse]

The current state of the Fil+ program demands immediate technical intervention due to the prevalent issue of fraud and misuse (12+ months ongoing). It's a well-known fact that the program's existing structure is being exploited to its maximum, with scammers successfully manipulating the system to acquire significant datacap allocations. Additionally, a substantial number of notaries within the program have been found to engage in fraudulent activities since their acceptance.

A short reminder: This. Will. Never. Stop. As long as we do not act.

We must acknowledge that the program's original pillars, notably the LDN pathway, simply do not work. To usher in a new era of excellence for Filecoin, it is imperative to steer away from the LDN model and the current notary system, as they no longer serve as viable solutions to these complex challenges safely. At least minimise their impact to the utmost minimum.

I'm really impressed with this proposal for the Fil+ program. It's clear that the team has put a lot of thought into addressing the program's limitations and ensuring it stays adaptable and responsive. The focus on tailored solutions for clients and the commitment to transparency are particularly commendable. It's great to see that they've thought about potential risks and have mitigation plans in place. Overall, this proposal feels like a positive step forward for Fil+ and Filecoin as a whole.

Well done to the team behind it! @Kevin-FF-USA @kevzak @dkkapur @simonkim0515 @raghavrmadya

[The-Wayvy]

tldr;

Miners! Please find new ways to simulate demand for storage so that we can give y’all unlimited Datacap.

Filecoin+ is an expensive and harmful charade.

[Randy998]

I support innovation, but what is the purpose of reform? At the current stage, the fundamental issue being discussed with Fil+ is corruption. How does this proposal address and resolve the issue of corruption? In my view, it doesn't actually solve problems but instead introduces additional risks. More allocators pathways clearly gonna cause more potential problems. I think we should focus on addressing existing problems rather than creating new ones.

[1475Notary]

" As the ecosystem has evolved, it has uncovered a myriad of new use cases which necessitate the introduction of fresh programs and pathways."

Can you share what these “new” use cases are? Before adding some new pathways, we need to figure out why the current process isn't working for them. Otherwise the discussion in Istanbul may be happening on the wrong assumptions.

[Fionasunmm]

I agree with the previous statement. This proposal does sound strange, and it makes me concerned about the potential additional issues this new approach might bring. I think we should postpone this proposal.

[galen-mcandrew]

Thanks for the comments! We will still be holding a new round of notary elections in the next few months, and sunsetting some existing PoC's and pathways. The LDN pathway has grown to encompass the majority of DataCap applications, and we are seeing that the rules & tools are not adequately serving the needs of the stakeholders. In an effort to grow the program, we are seeking to provide more specialization, rather than a 'one-size fits most' approach. Also, there are teams building new pathways to DataCap that do not fit in the current system (Direct Allocations, LDN v3.1, E-Fil+), along with new clients, notaries, and SPs that are looking for more specialized pathways. Some examples:

• market-based bidding mechanisms
• specialized tooling for open and retrievable datasets
• pathways with stricter guidelines around enterprise, encrypted, and SP-led business development
• teams supporting MSPs and ISVs
• additional automated diligence mechanisms
• organizations with a specific and narrow type of client and data, serving as onboarding & diligence partners

We look forward to seeing how organizations in the community, new and existing, apply to serve as notaries in the Filecoin Plus program.

[herrehesse]

As long as we sunset LDN pathway and remove all current notaries. If we do not do this the abuse will never stop.

[Harperling]

In an effort to grow the program, we are seeking to provide more specialization, rather than a 'one-size fits most' approach.

@herrehesse Galen just mentioned not to make ’one-size fits most' approach, and here you are suggesting to cancel all notaries. bit too arbitrary

[herrehesse]

@Harperling I do not argue with newly created account with zero reputation. But yes since almost all current notaries and LDN's are fraudulent, the only proper way forward is limiting or closing them.

There is no path forward keeping the current status quo.

[cryptoAmandaL]

I watched today's Fil+ YouTube livestream and gov team mentioned many details that might not be present in this post.

The lack of understanding from others could be due to incomplete information, so I hope to share more details to address people‘s concerns.

[spaceT9]

I got a question, if automated allocator is selected, no human involved, it's processed by computing. what is an allocator for?

[The-Wayvy]

@Filplus-govteam

Please publish all applications for community review

[MegaFil]

It sounds like fraud implementation requires the cooperation of at least two allocators before, but in the future it will only require one.

[herrehesse]

The plan is to automate the removal of abuse by a bot and closely monitor allocators allocations and performance and make community decisions on keeping or removing them. Much better than now with 60+ abusive notaries.

[psh0691]

Reforms can be accompanied by expected and other invisible problems. Centralization, inconsistency with FIL+ goals, laziness of allocators, fraud abuse, collusion, diversity turns into complexity... But, if we go toward a common goal, we can definitely solve the problem.