ProxyBroker 'freeze' (don't exit) if no limit specified

[datphyr]

I'm developing some application that constantly fetching API of specified sites. This sites have throttling on this API, but I need to fetch it really frequent. I found your ProxyBroker and this stuff is really great, but it's lacking some functionality in my particular case. I need mass of proxies to make, in ideal, every API call from different address. I made some experiments with ProxyBroker and here is what I found:

• 'serve' mode - can't use it cuz it's rotating proxies really slowly, I cannot find any settings to make rotation faster and I'm newb in Python to edit code by myself;
• I found a walkaround - I can stream ProxyBroker output to redis sorted set and use it by my puller workers, BUT ProxyBroker 'freezing' without setting limit (in my case ~450 proxies). I can walkaround this too by adding a timestamp to each found proxy and checking 'how fresh is last proxy', if it's for example 10 secs old - restart ProxyBroker.

Looks like ProxyBroker have internal list of sites where it grabbing proxies, and printing every found and tested proxy to stdout, when it's exhausted all this sites - it simply stops and not exits. Can you add functionality to 'start again after all sites parsed'? Or simply exit.

[datphyr]

Maybe I can even call it a real bug - 'ProxyBroker don't exit if no limit specified'. How to reproduce: 'proxybroker find --types HTTP HTTPS SOCKS4 SOCKS5 CONNECT:80 CONNECT:25' and wait :) Expected result: proxybroker exit Actual result: proxybroker found proxies and freeze

[datphyr]

Making more tests right now and looks like it don't appear if 'short type list' specified (like HTTP only).

[datphyr]

Nope ... I can confirm it happens all the time for me. Need some 'versions' report? I'm using ProxyBroker from pip3.

[akuzia]

Proxylist_me went bad, just comment it in providers list. Fixed for me

[gbballpack]

I can confirm the same issue. If I use the grabber feature and just try and pull a list of proxies without setting a limit, proxybroker makes a list but never exists or closes itself out. i.e., "proxybroker grab --outfile ./proxies.txt" Since proxybroker usually grabs around 9,000 proxies total, my work around has been to set of limit of 8500 results. i.e., "proxybroker grab --limit 8500 --outfile ./proxies.txt"