unclecode / crawl4ai

🔥🕷️ Crawl4AI: Open-source LLM Friendly Web Crawler & Scrapper
Apache License 2.0
16.42k stars 1.21k forks source link

Improve discoverability in chatGPT and other coding assistants #126

Open codelion opened 1 month ago

codelion commented 1 month ago

Hi,

Since this is a recent repository, if someone wants to generate code that uses this library in either chatGPT or any other coding assistant it doesn't work. Would it be possible to add a link to https://docs.codes/pypi/crawl4ai in your README.md somewhere to help users add just enough context about the library to use it with any coding assistant/chatGPT.

See the difference in the conversation with chatGPT

without docs.codes

Screenshot 2024-10-03 at 12 43 47 AM

v/s

with docs.codes

Screenshot 2024-10-03 at 12 45 54 AM
unclecode commented 1 month ago

Such an interesting suggestion. You're very right, and thank you for bringing it up. Perhaps what we're going to do is create GPTs that include all documents and proper instructions. We'll then release the GPTs for everyone who wants to create code with our library, and definitely add your equested links to our ReadMe as well. This has been part of our plan to create a chatbot and use crawl4AI to crawl its own website to answer people's queries. However, I think we can speed up the process by creating GPTs and allowing everyone to play around with them. Moreover we will create a markdown file specifically for attaching to chat with language models. Thank you so much again for bringing this up. We'll make it ready soon.