unclecode / crawl4ai

πŸ”₯πŸ•·οΈ Crawl4AI: Crawl Smarter, Faster, Freely. For AI.
https://crawl4ai.com
Apache License 2.0
17.01k stars 1.26k forks source link

Circular import error #130

Closed MSR2201 closed 1 month ago

MSR2201 commented 1 month ago

Traceback (most recent call last): File "/Users/youknowwho/Downloads/repos/ai/customllm/custom-llm/Automation_code/Web_crawling/crawl4ai.py", line 2, in import crawl4ai ^^^^^^^^^^^^^^^ File "/Users/youknowwho/Downloads/repos/ai/customllm/custom-llm/Automation_code/Web_crawling/crawl4ai.py", line 3, in from crawl4ai import AsyncWebCrawler as aicrawl4ai ImportError: cannot import name 'AsyncWebCrawler' from partially initialized module 'crawl4ai' (most likely due to a circular import) (/Users/youknowwho/Downloads/repos/ai/customllm/custom-llm/Automation_code/Web_crawling/crawl4ai.py)

while running the below ccode import asyncio from crawl4ai import AsyncWebCrawler async def main(): async with AsyncWebCrawler(verbose=True) as crawler: result = await crawler.arun(url="https://aurohosting.com/") print(result.markdown)

if name == "main": asyncio.run(main())

unclecode commented 1 month ago

@MSR2201 This seems weird to me. I just tested your code. There's no issue with it. Maybe you can try it again in the new versions we're going to release by tomorrow, 0.3.6. I hope this has been resolved already over there. I'll show you a sample of what happens when I execute your code. However, please test it and let me know. Thank you very much.

import asyncio
import crawl4ai
from crawl4ai import AsyncWebCrawler

async def main():
    async with AsyncWebCrawler(verbose=True) as crawler:
        result = await crawler.arun(url="https://aurohosting.com/", bypass_cache=True)
        print(result.markdown)

if __name__ == "__main__":
    asyncio.run(main())

Output:

[LOG] 🌀️  Warming up the AsyncWebCrawler
[LOG] 🌞 AsyncWebCrawler is ready to crawl
[LOG] πŸ•ΈοΈ Crawling https://aurohosting.com/ using AsyncPlaywrightCrawlerStrategy...
[LOG] βœ… Crawled https://aurohosting.com/ successfully!
[LOG] πŸš€ Crawling done for https://aurohosting.com/, success: True, time taken: 1.42 seconds
[LOG] πŸš€ Content extracted for https://aurohosting.com/, success: True, time taken: 0.06 seconds
[LOG] πŸ”₯ Extracting semantic blocks for https://aurohosting.com/, Strategy: AsyncWebCrawler
[LOG] πŸš€ Extraction done for https://aurohosting.com/, time taken: 0.07 seconds.
Skip to content

![Five Minutes](https://aurohosting.com/wp-content/uploads/2020/08/logo.png)

Technology Solutions in Five Minutes
....
MSR2201 commented 1 month ago

i have tested it and i got it i was using the crawl4ai as the file name for my code that made error because it is importing itself instead of crawl4ai library. the mistake is on my part