Closed MSR2201 closed 1 month ago
@MSR2201 This seems weird to me. I just tested your code. There's no issue with it. Maybe you can try it again in the new versions we're going to release by tomorrow, 0.3.6. I hope this has been resolved already over there. I'll show you a sample of what happens when I execute your code. However, please test it and let me know. Thank you very much.
import asyncio
import crawl4ai
from crawl4ai import AsyncWebCrawler
async def main():
async with AsyncWebCrawler(verbose=True) as crawler:
result = await crawler.arun(url="https://aurohosting.com/", bypass_cache=True)
print(result.markdown)
if __name__ == "__main__":
asyncio.run(main())
Output:
[LOG] π€οΈ Warming up the AsyncWebCrawler
[LOG] π AsyncWebCrawler is ready to crawl
[LOG] πΈοΈ Crawling https://aurohosting.com/ using AsyncPlaywrightCrawlerStrategy...
[LOG] β
Crawled https://aurohosting.com/ successfully!
[LOG] π Crawling done for https://aurohosting.com/, success: True, time taken: 1.42 seconds
[LOG] π Content extracted for https://aurohosting.com/, success: True, time taken: 0.06 seconds
[LOG] π₯ Extracting semantic blocks for https://aurohosting.com/, Strategy: AsyncWebCrawler
[LOG] π Extraction done for https://aurohosting.com/, time taken: 0.07 seconds.
Skip to content
![Five Minutes](https://aurohosting.com/wp-content/uploads/2020/08/logo.png)
Technology Solutions in Five Minutes
....
i have tested it and i got it i was using the crawl4ai as the file name for my code that made error because it is importing itself instead of crawl4ai library. the mistake is on my part
Traceback (most recent call last): File "/Users/youknowwho/Downloads/repos/ai/customllm/custom-llm/Automation_code/Web_crawling/crawl4ai.py", line 2, in
import crawl4ai
^^^^^^^^^^^^^^^
File "/Users/youknowwho/Downloads/repos/ai/customllm/custom-llm/Automation_code/Web_crawling/crawl4ai.py", line 3, in
from crawl4ai import AsyncWebCrawler as aicrawl4ai
ImportError: cannot import name 'AsyncWebCrawler' from partially initialized module 'crawl4ai' (most likely due to a circular import) (/Users/youknowwho/Downloads/repos/ai/customllm/custom-llm/Automation_code/Web_crawling/crawl4ai.py)
while running the below ccode import asyncio from crawl4ai import AsyncWebCrawler async def main(): async with AsyncWebCrawler(verbose=True) as crawler: result = await crawler.arun(url="https://aurohosting.com/") print(result.markdown)
if name == "main": asyncio.run(main())