search

mem0ai / mem0

The Memory layer for your AI apps
https://mem0.ai
Apache License 2.0
22.99k stars 2.12k forks source link

Nice elonmusk bot, but make him current: Please also support RAG for social media such as twitter #1124

Open yosun opened 10 months ago

yosun commented 10 months ago

🚀 The feature

Hey, Since you have elonmusk as your main readme example, please have that example also include social media such as twitter.com and instagram.com (etc)

import os
from embedchain import Pipeline as App

# Create a bot instance
os.environ["OPENAI_API_KEY"] = "YOUR API KEY"
elon_bot = App()

# Embed online resources
elon_bot.add("https://en.wikipedia.org/wiki/Elon_Musk")
elon_bot.add("https://www.forbes.com/profile/elon-musk")

# Query the bot
elon_bot.query("What's the latest with Elon Musk?") 

  File "/Users/mbp16_2022_m1max/MyTests/python/test_embedchain.py", line 11, in <module>
    elon_bot.add("https://www.x.com/elonmusk")
  File "/Users/mbp16_2022_m1max/MyTests/python/embedchainenv/lib/python3.9/site-packages/embedchain/embedchain.py", line 198, in add
    documents, metadatas, _ids, new_chunks = self._load_and_embed(
  File "/Users/mbp16_2022_m1max/MyTests/python/embedchainenv/lib/python3.9/site-packages/embedchain/embedchain.py", line 363, in _load_and_embed
    embeddings_data = chunker.create_chunks(loader, src, app_id=app_id, config=add_config.chunker)
  File "/Users/mbp16_2022_m1max/MyTests/python/embedchainenv/lib/python3.9/site-packages/embedchain/chunkers/base_chunker.py", line 31, in create_chunks
    data_result = loader.load_data(src)
  File "/Users/mbp16_2022_m1max/MyTests/python/embedchainenv/lib/python3.9/site-packages/embedchain/loaders/web_page.py", line 26, in load_data
    response.raise_for_status()
  File "/Users/mbp16_2022_m1max/MyTests/python/embedchainenv/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://twitter.com/elonmusk

Motivation, pitch

Nice elonmusk bot, but make him current: Please also support RAG for social media such as twitter etc

deshraj commented 10 months ago

Hello @yosun, thank you for raising this issue. Your suggestion to include social media platforms like Instagram and X (substitute for Twitter) in our elonmusk bot example is insightful. However, integrating these platforms have limitations on how much data you can fetch but this definitely doable.

We will try to prioritize this soon.