search

eyalzek / price-alert

Script to send an email when a price drops below a given value (tested on Amazon).
78 stars 34 forks source link

does it need proxy rotation for json items? #9

Closed artgod1 closed 5 years ago

artgod1 commented 5 years ago

Congrats for the script. It's working , but after checking about 100 items it gives the message: Didn't find the 'price' element.Trying again later. and it's emailing me for every other item without reason for alert. However, it's all working again in a few hours , so I guess amazon just temporary blocked the ip address. Any suggestions how to change the proxy between the JSON items? Many Thanks

eyalzek commented 5 years ago

Not sure what you mean by proxy rotation, but do you mind sharing your item list configuration so I can see reproduce? I never actually ran it with so many items, it makes sense that amazon has some kind of rate limiting on their side.

artgod1 commented 5 years ago

I am not allowed to share the large data file here , because it's based on a deep research but here you are about 40 items to test with (you can paste them a few time in the json) Also try to run it twice or three times and I am sure you will see that it will stop working . About proxy rotation, I meant something like : https://codelike.pro/create-a-crawler-with-rotating-ip-proxy-in-python/

["B008J2MZL8", 25.33], ["B002IS0M32", 11.99], ["B078CR92V5", 3.27], ["B01IDA940U", 19.99], ["B019WCXKGW", 24.73], ["B016XS2NM0", 37.49], ["B006PEBH0C", 12.01], ["B002B55BN8", 14.50], ["B0160OYPEQ", 38.95], ["B01H4ZC4YI", 14.99], ["B00YFPD76K", 24.97], ["B001KC05PM", 25.65], ["B002APJCNE", 15.99], ["B00AR9HX3G", 18.0], ["B01MT1LAOB", 1.5], ["B00SNM4KS0", 19.99], ["B00143XJ7I", 12.99], ["B07GXLLV3H", 10.19], ["B00NVMIO02", 39.99], ["B01FCMV5RM", 10.49], ["B01E2CI9HW", 10.99], ["B06XFJGV68", 35.0], ["B07C2XT63S", 5.59], ["B00BG6304A", 68.99], ["B07C55N9XW", 128.99], ["B073HTB3XV", 9.99], ["B078X979Z9", 8.99], ["B000GVTCSO", 6.39], ["B000I2Q0F4", 10.95], ["B0758F5GZJ", 29.99 ], ["B00129XP8M", 18.94 ], ["B00M35Y2MC", 37.97], ["B01GRYPGE2", 41.58], ["B01NCJMULG", 47.96], ["B01B56T45Y", 48.90], ["B073C54BCM", 15.75], ["B06XK5BCCY", 16.99], ["B0187EEUHO", 19.99], ["B003U6KZ06", 14.16], ["B006MRV4M8", 28.83],

eyalzek commented 5 years ago

Thanks, I'll test it when I have the time. Did you try implementing this random proxy with your config? PR's are welcome :)

artgod1 commented 5 years ago

I tried, but it didn't work for me.Probably I haven't done it correctly but If you can help with this ,it will be much appreciated. Many Thanks

eyalzek commented 5 years ago

I tried debugging this a bit, I do see a couple of places where it says Didn't find the 'price' element, trying again later, but that kind of make sense, in this case for example: https://www.amazon.de/dp/B01IDA940U there is no "default" price, only offers from the market place (which the script doesn't really support). Also, it shouldn't send emails in those cases, it should skip them and try them on the next run. Can you provide logs? Do you maybe run the script in multiple places with the same email credentials?

artgod1 commented 5 years ago

Thanks for your help. It makes sense for no default price.I sorted it out by changing the proxy and specifying another user agents otherwise I run into captchas ( Blocked ). You can close the issue now