Create a new guide about how to not get blocked

apify / crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

https://crawlee.dev/python/

Apache License 2.0

4.74k stars 322 forks source link

Create a new guide about how to not get blocked #481

Open vdusek opened 3 months ago

vdusek commented 3 months ago

We should create a new documentation guide about how to not get blocked.
Inspiration: https://crawlee.dev/docs/guides/avoid-blocking
This should be done once fingerprint-related issues are done (#401, #402, #549, ...).

29deepanshutyagi commented 2 months ago

i want to work on this issue @B4nan , kindly assign me ,if it's still opened

MostlyKIGuess commented 2 months ago

I want to work on this issue as well, I will try to finish by tonight. @souravjain540

iblameRishi commented 1 month ago

@vdusek I'd like to work on this, please assign it to me if it's still open

janbuchar commented 1 month ago

@vdusek I'd like to work on this, please assign it to me if it's still open

We don't assign issues for hacktoberfest. If you want to work on this, open a PR. First mergeable one gets merged.