search

jef / streetmerchant

🤖 The world's easiest, most powerful stock checker
https://jef.buzz/streetmerchant
MIT License
4.97k stars 1.3k forks source link

Navigation timeout of 20000 ms exceeded for BestBuy 3080 Founders Edition #2189

Closed rnovo1983 closed 3 years ago

rnovo1983 commented 3 years ago

Expected Behavior

I have tried running the app from a friends house and its app can detect out of stock instead of timing out

Current Behavior

Every time I try to scan for the BestBuy 3080 Founders Edition I get a Navigation timeout of 20000 ms

Steps to Reproduce

Not sure if they somehow limited my access to their site or if it an app bug. But it happens every time I run the app.

Environment

# ** All configuration variables are optional **
# Read https://github.com/jef/streetmerchant#customization for help on customizing this file
#############################################################################################

ASCII_BANNER=""
ASCII_COLOR=""
AUTO_ADD_TO_CART="true"
BROWSER_TRUSTED=""
COUNTRY="usa"
DESKTOP_NOTIFICATIONS=""
DISCORD_NOTIFY_GROUP=""
DISCORD_WEB_HOOK=""
EMAIL_PASSWORD=""
EMAIL_TO=""
EMAIL_USERNAME=""
HEADLESS=""
IN_STOCK_WAIT_TIME=""
LOG_LEVEL="info"
LOW_BANDWIDTH="true"
MAX_PRICE_SERIES_3060TI=""
MAX_PRICE_SERIES_3070=""
MAX_PRICE_SERIES_3080=""
MAX_PRICE_SERIES_3090=""
MAX_PRICE_SERIES_CORSAIR_SF=""
MAX_PRICE_SERIES_RX6800=""
MAX_PRICE_SERIES_RX6800XT=""
MAX_PRICE_SERIES_RX6900XT=""
MAX_PRICE_SERIES_RYZEN5600=""
MAX_PRICE_SERIES_RYZEN5800=""
MAX_PRICE_SERIES_RYZEN5900=""
MAX_PRICE_SERIES_RYZEN5950=""
MAX_PRICE_SERIES_SONYPS5C=""
MAX_PRICE_SERIES_SONYPS5DE=""
MAX_PRICE_SERIES_TEST=""
MICROCENTER_LOCATION=""
MQTT_BROKER_ADDRESS=""
MQTT_BROKER_PORT=""
MQTT_CLIENT_ID=""
MQTT_PASSWORD=""
MQTT_QOS=""
MQTT_TOPIC=""
MQTT_USERNAME=""
NVIDIA_ADD_TO_CART_ATTEMPTS=""
NVIDIA_SESSION_TTL=""
OPEN_BROWSER="true"
PAGE_BACKOFF_MIN=""
PAGE_BACKOFF_MAX=""
PAGE_SLEEP_MIN="1000"
PAGE_SLEEP_MAX="2000"
PAGE_TIMEOUT="20000"
PAGERDUTY_INTEGRATION_KEY=""
PAGERDUTY_SEVERITY=""
PHILIPS_HUE_API_KEY=""
PHILIPS_HUE_CLOUD_ACCESS_TOKEN=""
PHILIPS_HUE_CLOUD_CLIENT_ID=""
PHILIPS_HUE_CLOUD_CLIENT_SECRET=""
PHILIPS_HUE_CLOUD_REFRESH_TOKEN=""
PHILIPS_HUE_LAN_BRIDGE_IP=""
PHILIPS_HUE_LIGHT_COLOR=""
PHILIPS_HUE_LIGHT_IDS=""
PHILIPS_HUE_LIGHT_PATTERN=""
PHONE_CARRIER=""
PHONE_NUMBER=""
PLAY_SOUND="soundname.wav"
PROXY_ADDRESS=""
PROXY_PROTOCOL=""
PROXY_PORT=""
PUSHBULLET=""
PUSHOVER_TOKEN=""
PUSHOVER_USER=""
PUSHOVER_PRIORITY=""
SCREENSHOT=""
SHOW_ONLY_BRANDS="nvidia"
SHOW_ONLY_MODELS="founders edition"
SHOW_ONLY_SERIES="3080"
SLACK_CHANNEL=""
SLACK_TOKEN=""
SMTP_ADDRESS=""
SMTP_PORT=""
STORES="bestbuy"
TELEGRAM_ACCESS_TOKEN=""
TELEGRAM_CHAT_ID=""
TWILIO_ACCOUNT_SID=""
TWILIO_AUTH_TOKEN=""
TWILIO_FROM_NUMBER=""
TWILIO_TO_NUMBER=""
TWITCH_ACCESS_TOKEN=""
TWITCH_CHANNEL=""
TWITCH_CLIENT_ID=""
TWITCH_CLIENT_SECRET=""
TWITCH_REFRESH_TOKEN=""
TWITTER_ACCESS_TOKEN_KEY=""
TWITTER_ACCESS_TOKEN_SECRET=""
TWITTER_CONSUMER_KEY=""
TWITTER_CONSUMER_SECRET=""
TWITTER_TWEET_TAGS=""
USER_AGENT=""
WEB_PORT=""

Logs

[10:24:23 AM] error :: ✖ [bestbuy] nvidia 3080 founders edition - Navigation timeout of 20000 ms exceeded [10:24:45 AM] error :: ✖ [bestbuy] nvidia 3080 founders edition - Navigation timeout of 20000 ms exceeded

ShawTheatre commented 3 years ago

I was doing some troubleshooting on this tonight and here's what I came up with.

When allowing the browser ui to load (headless=false), I noticed that the api Best Buy urls weren't loading... at all (even when attempting to look at the source/networking information in dev tools). I tried everything I could to get the page to load in the browser - used the direct product URL (instead of the api shortcut), used http vs https, even changed my Proxy between Socks5 and HTTP. It wasn't until I removed the proxy entirely that I was able to correctly get results.

I don't know if that's because BBY is tracking proxies and banning them/not feeding them html when requested, or if there's something on the server side that doesn't allow transmission through proxy - but for now my work around is to hit these directly using the store specific proxy option.

In /streetmerchant directory, I use 'global.proxies' to define my proxies - and for each store that I want to hit directly, I create the store name file in the same directory (ex: 'bestbuy.proxies') and leave the file blank. The logic says the store specific proxy file will be called (if one exists) instead of the global - which seems to be the case.

There's still the risk that BBY - or one of the other retailers - will ban my primary IP, but as of tonight I don't know another way around it. Hope this helps a bit.

ste032 commented 3 years ago

Setting a non 0 value to RESTART_TIME in dotenv seemed to help:

RESTART_TIME=200000 browser gets restarted after milliseconds

ShawTheatre commented 3 years ago

I made the recommended adjustment but I'm still seeing the same "Navigation Timeout" result

scottaculous commented 3 years ago

I get the same behavior whenever I use proxies with amd. Still debugging to understand why,

klords commented 3 years ago

@rnovo1983 are you using proxies files?

github-actions[bot] commented 3 years ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 10 days

github-actions[bot] commented 3 years ago

This issue has been closed because it is stale. Reopen if necessary.