SpikeHD / AmazonMonitor

Amazon price checker, item monitor and stock monitor, all shoved into a Discord bot
GNU General Public License v3.0
221 stars 38 forks source link

[ERROR] 301 from Amazon.de #68

Closed wirdo89 closed 1 year ago

wirdo89 commented 3 years ago

Describe the issue:

Hello, I'm trying to get this to work with an proxylist but I can't get it to work. Tried multiple proxy providers with user/pass and IP authentication but no luck so far. Proxies are not flagged since a manual connect to any of them with browser works just fine on the URL.

Changing to proxyless works fine though.

Paste the contents of your config.json, excluding the token, here:

{ "prefix":"!", "token":"", "minutes_per_check":0.5, (lowered for testing) "search_response_ms": 30000, "url_params":{}, "guild_item_limit":5, "cache_limit": 10, "required_perms":["ADMINISTRATOR"], "tld":"de", "auto_cart_link":true, "debug_enabled":true }

Provide console output with debug mode enabled:

[WARNING] This message is not an error, and the bot is still running. [MESSAGE] Watchlist Loaded [MESSAGE] Puppeteer Launched [LOG] Checking item prices... [MESSAGE] Type: proxy [MESSAGE] Selected proxy URL: http://XXXXX:XXXXXXX@e1.p.webshare.io [MESSAGE] Waiting a couple seconds for JavaScript to load... [DEBUG] Got page in 3135ms [MESSAGE] Type: [DEBUG] Detected as a regular item [DEBUG] Full object: [DEBUG] { full_title: '', full_link: 'https://www.amazon.de/dp/B08LNWPYRS/??', asin: 'B08LNWPYRS', seller: '', price: '', symbol: '', shipping: 'Free with prime', rating: '', features: [], availability: '', image: 'https://via.placeholder.com/300x300.png?text=No+Image' } [WARNING] Detected 7 empty values. Could potentially mean bot was flagged

SpikeHD commented 3 years ago

Can you give me a short example of what your proxylist looks like? What problem specifically are you having? Like, are the requests failing or something?

Edit: Sorry just read through the debug output a bit more, no need to answer those. Do they work without user/pass authentication? Can you test the Amazon request using the proxy with something like cURL to see what specifically comes back?

wirdo89 commented 3 years ago

Thank-you for your response.

I'm not getting anything back when doing a curl with or without proxy on that URL. When I try it on amazon.de itself it shows the same thing with or without proxy. The proxies work with and without user/password yes. Tested both though. When testing with proxy on for example "https://www.amazon.de/-/en/gp/movers-and-shakers/?ref_=nav_em_ms_0_1_1_4" it works fine with CURL and it downloads some code.

c:\temp>curl https://www.amazon.de/-/en/gp/movers-and-shakers/?ref_=nav_em_ms_0_1_1_4 --proxy 185.250.XX.XXX:8085 --output test.txt % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 41789 0 41789 0 0 41789 0 --:--:-- 0:00:01 --:--:-- 31467

c:\temp>curl amazon.de --proxy 185.250.XX.XXX:8085

301 Moved Permanently

301 Moved Permanently


Server
c:\temp>curl amazon.de 301 Moved Permanently

301 Moved Permanently


Server
SpikeHD commented 3 years ago

I believe what's happening is that the response the bot is getting is the same as the one cURL is getting, which is the 301 error, which isn't very helpful in terms of getting Amazon data, obviously. I believe it's a redirect-following error, so I can look into it later.

SpikeHD commented 1 year ago

I can't say I have ever come across this error since my last comment. If this is still an issue, let me know!