masa-finance / masa-bittensor

Bittensor Subnet Config
https://masa.ai
MIT License
9 stars 11 forks source link

Feature: Implement Web Scraper, Discord, Telegram and/or Diariazation endpoints #244

Open teslashibe opened 1 week ago

teslashibe commented 1 week ago

Problem Statement

Currently, our system only has functional endpoints for Twitter data. Discord and Telegram endpoints are stubbed but not yet implemented. We need to expand our data sources to include these platforms to provide a more comprehensive service.

Acceptance Criteria

Additional Context

Proposed Solution

  1. Review the existing Twitter endpoint implementation
  2. Design and implement similar structures for Discord and Telegram
  3. Ensure proper error handling and rate limiting for the new endpoints
  4. Integrate the new endpoints with the existing validator infrastructure

Next Steps

  1. Analyze the specific requirements for Discord and Telegram APIs
  2. Design the data structure for the new endpoints
  3. Implement the new endpoints
  4. Create unit and integration tests
  5. Update the documentation to include the new endpoints

This feature will enhance our platform's capabilities and provide more diverse data sources for our users. Please discuss any questions or concerns about this feature in the comments.

Luka-Loncar commented 1 week ago

Blocked by spikes from the week of 9th September.