search

jef / streetmerchant

🤖 The world's easiest, most powerful stock checker
https://jef.buzz/streetmerchant
MIT License
4.97k stars 1.3k forks source link

Store request delay configuration isn't working #1775

Closed ScottPierce closed 3 years ago

ScottPierce commented 3 years ago

I have my STORES environment variable configured like the documentation suggests:

STORES=amazon:30000:50000,bestbuy,newegg:20000:30000,microcenter,bandh:30000:40000,nvidia

Expected Behavior

Requests to bandh for all products should be spread out by at least 30 to 40 seconds so I don't get rate limited

Current Behavior

Requests sometimes are happening less than 10 seconds apart, and I am getting rate limited

Steps to Reproduce

I'm running with the following command:

docker run -it --rm --env-file env.txt ghcr.io/jef/streetmerchant:latest

Environment

# ** All configuration variables are optional **
# Copy this file and name it as `dotenv` (without backticks)
# Read https://jef.codes/streetmerchant/getting-started/#customization for help on customizing this file
########################################################################################################

ASCII_BANNER=
ASCII_COLOR=
AUTO_ADD_TO_CART=
BROWSER_TRUSTED=
COUNTRY=usa
DESKTOP_NOTIFICATIONS=
DISCORD_NOTIFY_GROUP=
DISCORD_NOTIFY_GROUP_3060TI=
DISCORD_NOTIFY_GROUP_3070=
DISCORD_NOTIFY_GROUP_3080=
DISCORD_NOTIFY_GROUP_3090=
DISCORD_NOTIFY_GROUP_CORSAIR_SF=
DISCORD_NOTIFY_GROUP_RX6800=
DISCORD_NOTIFY_GROUP_RX6800XT=
DISCORD_NOTIFY_GROUP_RX6900XT=
DISCORD_NOTIFY_GROUP_RYZEN5600=
DISCORD_NOTIFY_GROUP_RYZEN5800=
DISCORD_NOTIFY_GROUP_RYZEN5900=
DISCORD_NOTIFY_GROUP_RYZEN5950=
DISCORD_NOTIFY_GROUP_SONYPS5C=
DISCORD_NOTIFY_GROUP_SONYPS5DE=
DISCORD_NOTIFY_GROUP_XBOXSX=
DISCORD_NOTIFY_GROUP_XBOXSS=
DISCORD_NOTIFY_GROUP_TEST=
DISCORD_WEB_HOOK=
EMAIL_PASSWORD=
EMAIL_TO=
EMAIL_USERNAME=
HEADLESS=
IN_STOCK_WAIT_TIME=
LOG_LEVEL=
LOW_BANDWIDTH=true
MAX_PRICE_SERIES_3060TI=
MAX_PRICE_SERIES_3070=
MAX_PRICE_SERIES_3080=750
MAX_PRICE_SERIES_3090=
MAX_PRICE_SERIES_CORSAIR_SF=
MAX_PRICE_SERIES_RX6800=
MAX_PRICE_SERIES_RX6800XT=
MAX_PRICE_SERIES_RX6900XT=
MAX_PRICE_SERIES_RYZEN5600=
MAX_PRICE_SERIES_RYZEN5800=
MAX_PRICE_SERIES_RYZEN5900=
MAX_PRICE_SERIES_RYZEN5950=
MAX_PRICE_SERIES_DARKHERO=460
MAX_PRICE_SERIES_SONYPS5C=
MAX_PRICE_SERIES_SONYPS5DE=
MAX_PRICE_SERIES_XBOXSS=
MAX_PRICE_SERIES_XBOXSX=
MAX_PRICE_SERIES_TEST=
MICROCENTER_LOCATION=fairfax
MQTT_BROKER_ADDRESS=
MQTT_BROKER_PORT=
MQTT_CLIENT_ID=
MQTT_PASSWORD=
MQTT_QOS=
MQTT_TOPIC=
MQTT_USERNAME=
NVIDIA_ADD_TO_CART_ATTEMPTS=
NVIDIA_SESSION_TTL=
OPEN_BROWSER=
PAGE_BACKOFF_MIN=
PAGE_BACKOFF_MAX=
PAGE_SLEEP_MIN=
PAGE_SLEEP_MAX=
PAGE_TIMEOUT=
PAGERDUTY_INTEGRATION_KEY=
PAGERDUTY_SEVERITY=
PHILIPS_HUE_API_KEY=
PHILIPS_HUE_CLOUD_ACCESS_TOKEN=
PHILIPS_HUE_CLOUD_CLIENT_ID=
PHILIPS_HUE_CLOUD_CLIENT_SECRET=
PHILIPS_HUE_CLOUD_REFRESH_TOKEN=
PHILIPS_HUE_LAN_BRIDGE_IP=
PHILIPS_HUE_LIGHT_COLOR=
PHILIPS_HUE_LIGHT_IDS=
PHILIPS_HUE_LIGHT_PATTERN=
PHONE_CARRIER=
PHONE_NUMBER=
PLAY_SOUND=
PROXY_ADDRESS=
PROXY_PROTOCOL=
PROXY_PORT=
PUSHBULLET=
PUSHOVER_EXPIRE=
PUSHOVER_RETRY=
PUSHOVER_TOKEN=
PUSHOVER_USER=
PUSHOVER_PRIORITY=
SCREENSHOT=
SHOW_ONLY_BRANDS=asus,nvidia
SHOW_ONLY_MODELS=
SHOW_ONLY_SERIES=darkhero,sonyps5de,3080
SLACK_CHANNEL=
SLACK_TOKEN=
SMTP_ADDRESS=
SMTP_PORT=
STORES=amazon:30000:50000,bestbuy,newegg:20000:30000,microcenter,bandh:30000:40000,nvidia
TELEGRAM_ACCESS_TOKEN=
TELEGRAM_CHAT_ID=
TWITCH_ACCESS_TOKEN=
TWITCH_CHANNEL=
TWITCH_CLIENT_ID=
TWITCH_CLIENT_SECRET=
TWITCH_REFRESH_TOKEN=
TWITTER_ACCESS_TOKEN_KEY=
TWITTER_ACCESS_TOKEN_SECRET=
TWITTER_CONSUMER_KEY=
TWITTER_CONSUMER_SECRET=
TWITTER_TWEET_TAGS=
WEB_PORT=

Logs

[9:26:45 PM] info :: ✖ [microcenter] [asus (3080)] tuf :: PRICE 839.99 EXCEEDS LIMIT 750
[9:26:47 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:26:48 PM] info :: ✖ [microcenter] [asus (3080)] strix oc :: PRICE 929.99 EXCEEDS LIMIT 750
[9:26:50 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:26:51 PM] info :: ✖ [microcenter] [asus (darkhero)] crosshair viii :: OUT OF STOCK
[9:26:53 PM] info :: ✖ [bestbuy] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:26:59 PM] info :: ✖ [bestbuy] [asus (3080)] strix :: OUT OF STOCK
[9:27:00 PM] info :: ✖ [microcenter] [asus (3080)] tuf :: PRICE 839.99 EXCEEDS LIMIT 750
[9:27:01 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:02 PM] info :: ✖ [microcenter] [asus (3080)] strix oc :: PRICE 929.99 EXCEEDS LIMIT 750
[9:27:04 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:05 PM] info :: ✖ [microcenter] [asus (darkhero)] crosshair viii :: OUT OF STOCK
[9:27:07 PM] info :: ✖ [newegg] [asus (3080)] tuf :: OUT OF STOCK
[9:27:11 PM] info :: ✖ [bestbuy] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:11 PM] info :: ✖ [newegg] [asus (3080)] strix :: OUT OF STOCK
[9:27:15 PM] info :: ✖ [newegg] [asus (3080)] tuf oc :: OUT OF STOCK
[9:27:15 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:16 PM] info :: ✖ [bestbuy] [asus (3080)] strix :: OUT OF STOCK
[9:27:17 PM] info :: ✖ [bandh] [asus (3080)] tuf oc :: OUT OF STOCK
[9:27:18 PM] info :: ✖ [microcenter] [asus (3080)] tuf :: PRICE 839.99 EXCEEDS LIMIT 750
[9:27:19 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:20 PM] info :: ✖ [microcenter] [asus (3080)] strix oc :: PRICE 929.99 EXCEEDS LIMIT 750
[9:27:21 PM] info :: ✖ [bandh] [asus (3080)] strix oc :: OUT OF STOCK
[9:27:23 PM] info :: ✖ [microcenter] [asus (darkhero)] crosshair viii :: OUT OF STOCK
[9:27:24 PM] info :: ✖ [bandh] [asus (3080)] strix white :: OUT OF STOCK
[9:27:27 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:27 PM] info :: ✖ [bandh] [asus (3080)] ekwb :: OUT OF STOCK
[9:27:30 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:31 PM] info :: ✖ [amazon] [asus (3080)] tuf oc :: OUT OF STOCK
[9:27:32 PM] info :: ✖ [bestbuy] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:32 PM] info :: ✖ [microcenter] [asus (3080)] tuf :: PRICE 839.99 EXCEEDS LIMIT 750
[9:27:35 PM] info :: ✖ [microcenter] [asus (3080)] strix oc :: PRICE 929.99 EXCEEDS LIMIT 750
[9:27:37 PM] info :: ✖ [bestbuy] [asus (3080)] strix :: OUT OF STOCK
[9:27:38 PM] info :: ✖ [microcenter] [asus (darkhero)] crosshair viii :: OUT OF STOCK
[9:27:38 PM] info :: ✖ [amazon] [asus (3080)] tuf :: OUT OF STOCK
[9:27:40 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:43 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:44 PM] info :: ✖ [newegg] [asus (darkhero)] crosshair viii :: PRICE 849.99 EXCEEDS LIMIT 460
[9:27:44 PM] info :: ✖ [amazon] [asus (3080)] strix oc :: OUT OF STOCK
[9:27:46 PM] info :: ✖ [microcenter] [asus (3080)] tuf :: PRICE 839.99 EXCEEDS LIMIT 750
[9:27:49 PM] info :: ✖ [microcenter] [asus (3080)] strix oc :: PRICE 929.99 EXCEEDS LIMIT 750
[9:27:51 PM] info :: ✖ [microcenter] [asus (darkhero)] crosshair viii :: OUT OF STOCK
[9:27:53 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:53 PM] info :: ✖ [bestbuy] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:56 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:27:58 PM] info :: ✖ [bestbuy] [asus (3080)] strix :: OUT OF STOCK
[9:28:00 PM] info :: ✖ [microcenter] [asus (3080)] tuf :: PRICE 839.99 EXCEEDS LIMIT 750
[9:28:03 PM] info :: ✖ [bandh] [asus (3080)] tuf oc :: OUT OF STOCK
[9:28:03 PM] info :: ✖ [microcenter] [asus (3080)] strix oc :: PRICE 929.99 EXCEEDS LIMIT 750
[9:28:05 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:28:06 PM] info :: ✖ [microcenter] [asus (darkhero)] crosshair viii :: OUT OF STOCK
[9:28:06 PM] info :: ✖ [bandh] [asus (3080)] strix oc :: OUT OF STOCK
[9:28:08 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:28:09 PM] info :: ✖ [bandh] [asus (3080)] strix white :: OUT OF STOCK
[9:28:13 PM] info :: ✖ [bestbuy] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:28:13 PM] info :: ✖ [bandh] [asus (3080)] ekwb :: OUT OF STOCK
[9:28:13 PM] info :: ✖ [newegg] [asus (3080)] tuf :: OUT OF STOCK
[9:28:14 PM] info :: ✖ [microcenter] [asus (3080)] tuf :: PRICE 839.99 EXCEEDS LIMIT 750
[9:28:16 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:28:17 PM] info :: ✖ [newegg] [asus (3080)] strix :: OUT OF STOCK
[9:28:17 PM] info :: ✖ [microcenter] [asus (3080)] strix oc :: PRICE 929.99 EXCEEDS LIMIT 750
[9:28:18 PM] info :: ✖ [bestbuy] [asus (3080)] strix :: OUT OF STOCK
[9:28:19 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:28:20 PM] info :: ✖ [microcenter] [asus (darkhero)] crosshair viii :: OUT OF STOCK
[9:28:21 PM] info :: ✖ [newegg] [asus (3080)] tuf oc :: OUT OF STOCK
[9:28:24 PM] info :: ✖ [amazon] [asus (3080)] tuf oc :: OUT OF STOCK
[9:28:25 PM] info :: ✖ [newegg] [asus (darkhero)] crosshair viii :: PRICE 849.99 EXCEEDS LIMIT 460
[9:28:28 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:28:31 PM] info :: ✖ [amazon] [asus (3080)] tuf :: OUT OF STOCK
[9:28:31 PM] info :: ✖ [nvidia] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:28:32 PM] info :: ✖ [bestbuy] [nvidia (3080)] founders edition :: OUT OF STOCK
[9:28:33 PM] info :: ✖ [microcenter] [asus (3080)] tuf :: PRICE 839.99 EXCEEDS LIMIT 750
[9:28:35 PM] info :: ✖ [microcenter] [asus (3080)] strix oc :: PRICE 929.99 EXCEEDS LIMIT 750
[9:28:37 PM] info :: ✖ [bestbuy] [asus (3080)] strix :: OUT OF STOCK
[9:28:38 PM] info :: ✖ [amazon] [asus (3080)] strix oc :: OUT OF STOCK
[9:28:38 PM] info :: ✖ [microcenter] [asus (darkhero)] crosshair viii :: OUT OF STOCK
jxjftw commented 3 years ago

I believe the delay is per-item @ that store.

PAGE_SLEEP_MIN | Minimum sleep time between queries of the same product page. In milliseconds, default: 5000

PAGE_SLEEP_MAX | Maximum sleep time between queries of the same product page. In milliseconds, default: 10000
ScottPierce commented 3 years ago

What's the best way for me to reduce spamming a specific store then?

xkuza commented 3 years ago

STORES="amazon-de:10000:30000,amazon-fr:10000:30000"

github-actions[bot] commented 3 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

github-actions[bot] commented 3 years ago

This issue has been closed because it is stale. Reopen if necessary.