Model Based Detection

[erichfi]

Model Based Detection API Roadmap

Objective: This roadmap outlines the strategic plan for the phased expansion of the Model Based Verification API. The aim is to integrate additional models to provide nuanced verification services, allowing developers to verify user integrity without direct user action.

Current Implementation:

• Ethereum Activity Model: Analyzes Ethereum mainnet transactions to differentiate between human and potential Sybil entities.

Planned Models:

• zkSync Model: Focuses on Layer 2 activity, specifically zkSync transactions.
• NFT Model: Evaluates transaction patterns related to Eth mainnet NFT interactions.
• OP Model: Focuses on Layer 2 activity, specifically OP transactions.
• Polygon Model: Focuses on Sidechain / Layer 2 activity, specifically Polygon transactions.

Later:

• ARB Model: Focuses on Layer 2 activity, specifically ARB transactions.

Issues to Include:

1. Integrate zkSync and NFT Scoring Models into API Analysis Endpoint

   • Issue ID: #2459
   • Description: Integrate zkSync and NFT scoring models into the existing API analysis endpoint to enhance verification capabilities.
   • Include OP and Polygon too.
   • Maybe have a parameter to specify which models you care about
   • Rate limit per request or per model request?

2. Evaluate Operational Costs for zkSync and NFT Scoring Integration in API

   • Issue ID: #2460
   • Description: Analyze the operational costs associated with the integration of zkSync and NFT scoring models into the API to ensure cost-effectiveness and sustainability.

3. Setting up Separate Rate Limits for the Endpoint

   • Issue ID: https://github.com/gitcoinco/passport/issues/2540
   • Description: Implement rate limits for the new endpoint to manage usage effectively while allowing flexibility for future adjustments. The initial rate limit should be reasonable, with potential for future adjustments. For example, X calls per day or Y calls per month.
   • Separate ratelimits on MBD from rest of APIs.
   • Stricter rate limits on MBD with two tiers.
     • @lebraat two put together recommendations for the lowest limit and higher limit.
     • will communicate costs of api call per model

Considerations and Best Practices:

• Rate Limiting Strategy:

  • Initial Setup: Implement a rate limit that balances usability and resource management. Start with a moderate limit and adjust based on user feedback and system performance.
  • Future Adjustments: Plan for the ability to modify rate limits as needed to accommodate growing usage and evolving requirements.

• Monetization Strategy:

  • Future Pricing Models: Develop a pricing strategy for API usage that can evolve from free tiers to paid plans. Options could include pay-per-call or subscription-based models.
  • Usage Monitoring: Implement mechanisms to monitor API usage to inform pricing strategy and ensure fair usage.

• Common API Methods in Web3:

  • Methodology: Research and document common methods used in Web3 for similar integrations to ensure our API remains competitive and meets developer expectations.

• Meta Models:

  • Composite Scoring: Explore the feasibility of meta models that aggregate scores from various individual models to provide a comprehensive user evaluation.
  • Payload Integration: Consider including meta model outputs within the existing endpoint payload, either within the 'models' section of the JSON or as a separate category.

• Bulk Address Processing:

  • System Efficiency: Propose a system for batch processing of addresses to optimize system efficiency and avoid real-time overloads.
  • Developer Notification: Implement email notifications to inform developers when batch processing is complete, improving user experience and system performance.

• Documentation and Support:

  • Comprehensive Guides: Develop detailed documentation and tutorials to support developers in using the API effectively.
  • Feedback Channels: Establish continuous feedback mechanisms to gather insights from developers and refine the API based on real-world usage.

Open Questions:

• Meta Model Integration: Evaluate the technical and economic feasibility of implementing meta models and their impact on payload management and computational overhead.
• Endpoint Structure: Decide between maintaining a unified endpoint with additional models via parameters or creating individual endpoints for each new model. Analyze the impacts on cost, efficiency, and developer ease of use.

Timeline and Milestones:

• Development and Integration: Continue the integration of zkSync and NFT models with proposed initial enhancements evaluated for potential rollout by June.

Collaboration and Feedback:

• Stakeholder Engagement: Engage regularly with stakeholders to discuss progress, address concerns, and align strategies for future developments.
• Developer Relations: Collaborate closely with the DevRel team to ensure developer needs for documentation and support are effectively met.
• Feedback Collection: Use surveys, interviews, and usage analytics to gather comprehensive feedback from developers.

[Jkd-eth]

@erichfi per a conversation with @lebraat today. I think we should add the NFT & L2 activity models to the endpoint sooner than later, so what we can test them as part of the 'beta' and then determine changes to be made + the aggregate score or model after

[erichfi]

@Jeremy-Gitcoin, acknowledged. Let's discuss more.

[erichfi]

Issues to include:

• 2459

• 2460

• Setting up separate rate limits for this endpoint
  • need: some rate limit that's not crazy low but still sets the epxectation that we might change it later. x calls a day or y calls a months.
  • later we wanna charge so we can set and monitor the number of calls and charge a fee, like price per call or packaging on a per month basis. what are the common api methods in web3?
• We might have Polygon, Arbitrum, and the Metamodel as well

[lebraat]

Hey team!

Had a chat with Kyle today and came up with a baseline rate limit that we are both pretty comfortable with.

I recommend that we also introduce a couple additional rate limits so that we can keep partners on a short leash if they want an elevation. I believe this will help us as we test out different pricing options, by not letting partners get used to high limits.

Tier 2 and 3 are the same as the higher tiers that we've set for the Passport API, while the base tier is set to 50 requests per 15 minutes. I feel like tier 2 would be great for smaller campaigns, while tier 3 would effectively support larger.

Tier	Rate limit	Maximum requests per day
Base Tier	50 requests per 15 minutes	4800 requests per day
Tier 2	350 requests per 15 minutes	33600 requests per day
Tier 3	2000 requests per 15 minutes	192000 requests per day

One key requirement will be that these rate limits will be completely separate from the Passport API.

Thoughts?

[erichfi]

At least for internal purposes, we should get a bulk scoring function into model based detection APIs. E.g, submit CSV and then get the scores back.