TeamHG-Memex scrapy-rotating-proxies issues

TeamHG-Memex / scrapy-rotating-proxies

use multiple proxies with Scrapy

MIT License

732 stars 156 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Use Queue to selec the next proxy

#70 devfox-se opened 4 months ago
0
how to use proxy authentication username and password

#69 zcafetoday opened 1 year ago
1
Python gt 3.6

#68 matiskay closed 2 years ago
0
How to Update the proxy_list.

#67 Kamranbarlas opened 2 years ago
0
it keeps checking my proxy servers Dead or not than actually doing its job of scraping data

#66 vijaykurhade opened 2 years ago
1
how to ban the proxy with timeout?

#65 leolle opened 2 years ago
1
Proxies working for longer time slows scraping down.

#64 Kamranbarlas opened 2 years ago
2
Where is the settings.py? Should I create a new one for me?

#63 alimertkoc opened 2 years ago
1
Read proxy list from an URL

#62 datawookie opened 2 years ago
4
Is there a way to bypass the proxy check entirely? Everything Slowed

#61 doverradio opened 3 years ago
0
Please add ability to not assign proxy that is in-use

#60 iamumairayub opened 3 years ago
1
Proxy-Authorization added

#59 dtalkachou opened 3 years ago
8
AttributeError: Response content isn't text in "response_is_ban"

#58 iamumairayub opened 3 years ago
1
Make typing dependency conditional < python 3.5

#57 yadalik opened 3 years ago
0
RETRY_HTTP_CODES is not taken into account

#56 caffeinatedMike opened 3 years ago
0
scrapy with splash and rotating proxies

#55 mianwaqas94 opened 3 years ago
1
Crawlera → Zyte Smart Proxy Manager

#54 Gallaecio opened 3 years ago
1
Update proxy list once a day

#53 mo-tech55 opened 3 years ago
1
Dependency `typing` breaks `pip install`

#52 StasDeep opened 3 years ago
3
Shouldn't an IgnoreRequest being raised if max_proxies_to_try is reached

#51 codekoriko opened 3 years ago
0
This middleware breaks default throttling

#50 3hhh opened 3 years ago
0
Using get_random() to select a proxy is not optimal

#49 fredd-427 opened 3 years ago
0
Banned responses get to the engine in the end

#48 3hhh opened 3 years ago
0
Insensible middleware priority recommendations

#47 3hhh opened 3 years ago
2
Only rotate proxy when banned or dead

#46 Marow opened 4 years ago
0
None of the Proxies are checked - leading to perpetual process where scraping never starts

#45 caffeinatedMike opened 4 years ago
0
stop rotating proxy

#44 MaziarTalebi opened 4 years ago
0
integrate middleware with standalone Scrapy script

#43 beanben closed 4 years ago
1
Clean proxy in logs

#42 StasDeep opened 4 years ago
2
Log proxy stats only if ROTATING_PROXY_LOGSTATS_INTERVAL is non-zero

#41 StasDeep closed 4 years ago
3
Refresh the list of proxies during scraping

#40 dibodin opened 4 years ago
11
Possibility to use proxies from DB?

#39 fparareda opened 4 years ago
1
Add dynamic signaling

#38 n1kn4x opened 4 years ago
2
raise error.ReactorNotRestartable() twisted.internet.error.ReactorNotRestartable

#37 Arregator closed 4 years ago
1
Proxies Info stucked so much times

#36 Marow opened 4 years ago
0
Add Support for Priority Proxies

#35 iamashks opened 4 years ago
1
All proxies unchecked except one

#34 milicamilivojevic opened 4 years ago
4
Fix the setting of `Proxy-Authorization` header

#33 tianhuil opened 4 years ago
0
retrying request after ban does not respect DOWNLOAD_DELAY and autothrottle

#32 kakarukeys opened 4 years ago
0
What about getting proxy list from url?

#31 aidiss opened 4 years ago
2
Strip away password and/or username from proxy url before sending them to logs

#30 umrashrf opened 5 years ago
0
about proxy without scheme(protocol)

#29 searchlyf opened 5 years ago
1
Problem using proxy with scrapy

#28 Nizarazo opened 5 years ago
1
Retry Request

#27 kadimon opened 5 years ago
1
WIP: add BanAwareCachePolicy for http caching

#26 Granitosaurus opened 5 years ago
1
Always return float value for mean_backoff_time

#25 tlinhart closed 5 years ago
2
fix bug with good proxy stats

#24 Granitosaurus closed 5 years ago
3
DOWNLOAD_DELAY will not work if use proxy.

#23 dodoflyy opened 5 years ago
1
Does it works with https or socks proxy?

#22 ghostku closed 5 years ago
1
add stats tracking to proxy middleware

#21 Granitosaurus closed 5 years ago
5