Changed the format of proxies.json to just a list to simplify for myself.
In AmazonMonitoringHandler's init, each proxy is placed in a nested dictionary with the idx number as the parent key and the inner dict holding proxy as key and time.time() as value.
eg) {31: {user:pw@ip:port : time.time()}}
The function AmazonMonitor.get_new_proxy() is ran in each stock_check function where it uses randint to generate a number between 0 and (number of proxies - 1), which is used to call the dictionary holding the respective proxy address and the last time it was used. All of this is in a while loop that breaks when the condition that the last access time of the time value is greater than 6 seconds.
It's mega clunky and I still don't have a very clear idea how async works and would love some advices to improve or change the design of this. Thanks.
Tried adding random rotation to the proxies.
Changed the format of
proxies.jsonto just a list to simplify for myself.In
AmazonMonitoringHandler's init, each proxy is placed in a nested dictionary with theidxnumber as the parent key and the inner dict holding proxy as key andtime.time()as value. eg) {31: {user:pw@ip:port : time.time()}}The function
AmazonMonitor.get_new_proxy()is ran in eachstock_checkfunction where it usesrandintto generate a number between 0 and (number of proxies - 1), which is used to call the dictionary holding the respective proxy address and the last time it was used. All of this is in a while loop that breaks when the condition that the last access time of the time value is greater than 6 seconds.It's mega clunky and I still don't have a very clear idea how async works and would love some advices to improve or change the design of this. Thanks.