search

jef / streetmerchant

🤖 The world's easiest, most powerful stock checker
https://jef.buzz/streetmerchant
MIT License
4.97k stars 1.3k forks source link

Price filter not working (maybe caused for numerical representation of the price) #1676

Closed liam-maps closed 3 years ago

liam-maps commented 3 years ago

Expected Behavior

The filter should work if the price is higher than the price I've set

Current Behavior

The log shows a found product, even if the price is higher

Steps to Reproduce

This product is over the thousands tag, but the thousands separator is a space, which I suspect is causing the issue https://www.wipoid.com/asus-rog-strix-geforce-rtx-3080-oc-edition-10gb-gddr6x.html

With this also happens, but in this case the thousands separator is a dot, and the cents separator is a comma, because in Spain and other countries this is how it goes, which I suspect is causing the issue https://www.vsgamers.es/product/tarjeta-grafica-asus-tuf-gaming-geforce-rtx-3090-oc-24g-24-gb-gddr6x

EDIT: Over thousand price triggered the same behaviour again, here: https://www.amazon.es/dp/B08HM4V2DH

I don't know if this is the culprit or if you are taking into account all of the separator variants around the globe: https://en.wikipedia.org/wiki/Decimal_separator#Examples_of_use

Environment

Kubuntu 19.10

# ** All configuration variables are optional **
# Read https://github.com/jef/streetmerchant#customization for help on customizing this file
#############################################################################################

ASCII_BANNER=""
ASCII_COLOR=""
AUTO_ADD_TO_CART=""
BROWSER_TRUSTED=""
COUNTRY="spain"
DESKTOP_NOTIFICATIONS=""
DISCORD_NOTIFY_GROUP=""
DISCORD_WEB_HOOK=""
EMAIL_PASSWORD=""
EMAIL_TO=""
EMAIL_USERNAME=""
HEADLESS=""
IN_STOCK_WAIT_TIME=""
LOG_LEVEL=""
LOW_BANDWIDTH="true"
MAX_PRICE_SERIES_3060TI="550"
MAX_PRICE_SERIES_3070="650"
MAX_PRICE_SERIES_3080="750"
MAX_PRICE_SERIES_3090="1650"
MAX_PRICE_SERIES_CORSAIR_SF=""
MAX_PRICE_SERIES_RX6800=""
MAX_PRICE_SERIES_RX6800XT=""
MAX_PRICE_SERIES_RX6900XT=""
MAX_PRICE_SERIES_RYZEN5600=""
MAX_PRICE_SERIES_RYZEN5800=""
MAX_PRICE_SERIES_RYZEN5900=""
MAX_PRICE_SERIES_RYZEN5950=""
MAX_PRICE_SERIES_SONYPS5C=""
MAX_PRICE_SERIES_SONYPS5DE=""
MAX_PRICE_SERIES_TEST=""
MICROCENTER_LOCATION=""
MQTT_BROKER_ADDRESS=""
MQTT_BROKER_PORT=""
MQTT_CLIENT_ID=""
MQTT_PASSWORD=""
MQTT_QOS=""
MQTT_TOPIC=""
MQTT_USERNAME=""
NVIDIA_ADD_TO_CART_ATTEMPTS=""
NVIDIA_SESSION_TTL=""
OPEN_BROWSER="false"
PAGE_BACKOFF_MIN=""
PAGE_BACKOFF_MAX=""
PAGE_SLEEP_MIN="5000"
PAGE_SLEEP_MAX="10000"
PAGE_TIMEOUT=""
PAGERDUTY_INTEGRATION_KEY=""
PAGERDUTY_SEVERITY=""
PHILIPS_HUE_API_KEY=""
PHILIPS_HUE_CLOUD_ACCESS_TOKEN=""
PHILIPS_HUE_CLOUD_CLIENT_ID=""
PHILIPS_HUE_CLOUD_CLIENT_SECRET=""
PHILIPS_HUE_CLOUD_REFRESH_TOKEN=""
PHILIPS_HUE_LAN_BRIDGE_IP=""
PHILIPS_HUE_LIGHT_COLOR=""
PHILIPS_HUE_LIGHT_IDS=""
PHILIPS_HUE_LIGHT_PATTERN=""
PHONE_CARRIER=""
PHONE_NUMBER=""
PLAY_SOUND=""
PROXY_ADDRESS=""
PROXY_PROTOCOL=""
PROXY_PORT=""
PUSHBULLET=""
PUSHOVER_TOKEN=""
PUSHOVER_USER=""
PUSHOVER_PRIORITY=""
SCREENSHOT=""
SHOW_ONLY_BRANDS=""
SHOW_ONLY_MODELS=""
SHOW_ONLY_SERIES="3070, 3090, 3080, 3060ti"
SLACK_CHANNEL=""
SLACK_TOKEN=""
SMTP_ADDRESS=""
SMTP_PORT=""
STORES="pccomponentes, amazon-es, coolmod, elcorteingles, wipoid, vsgamers"
TELEGRAM_ACCESS_TOKEN=""
TELEGRAM_CHAT_ID=""
TWILIO_ACCOUNT_SID=""
TWILIO_AUTH_TOKEN=""
TWILIO_FROM_NUMBER=""
TWILIO_TO_NUMBER=""
TWITCH_ACCESS_TOKEN=""
TWITCH_CHANNEL=""
TWITCH_CLIENT_ID=""
TWITCH_CLIENT_SECRET=""
TWITCH_REFRESH_TOKEN=""
TWITTER_ACCESS_TOKEN_KEY=""
TWITTER_ACCESS_TOKEN_SECRET=""
TWITTER_CONSUMER_KEY=""
TWITTER_CONSUMER_SECRET=""
TWITTER_TWEET_TAGS=""
USER_AGENT=""
WEB_PORT=""

Logs

wofnull commented 3 years ago

Your config has no max_prices set:

MAX_PRICE_SERIES_3060TI="" MAX_PRICE_SERIES_3070="" MAX_PRICE_SERIES_3080="" MAX_PRICE_SERIES_3090="" MAX_PRICE_SERIES_CORSAIR_SF="" MAX_PRICE_SERIES_RX6800="" MAX_PRICE_SERIES_RX6800XT="" MAX_PRICE_SERIES_RX6900XT="" MAX_PRICE_SERIES_RYZEN5600="" MAX_PRICE_SERIES_RYZEN5800="" MAX_PRICE_SERIES_RYZEN5900="" MAX_PRICE_SERIES_RYZEN5950="" MAX_PRICE_SERIES_SONYPS5C="" MAX_PRICE_SERIES_SONYPS5DE="" MAX_PRICE_SERIES_TEST=""

Therefore, it assumes there is no max price, and gives every item which is IN STOCK a mark that its available under buyprice.

wofnull commented 3 years ago

i let the script run for wipoid and vsgamers:

Wipoids Config seems most likely out of date, since many cards that are giving responses on the site are not directly listed anymore, however, no card that is configured is reported as instock and > 1000 € ... therefore i was not able to check yet the pricing feature.

checking against vsgamers brought another issue up: vsgamers does not check for max price at all, therefore anything that is available, will get reported as instock and put out by the bot via opening up the page.

For the vsgamers one, I already put a patch up for jef to add. However ... the wipoid one seems to take a bit more effort, since this will take a bit mor time to fix.

liam-maps commented 3 years ago

Your config has no max_prices set:

MAX_PRICE_SERIES_3060TI="" MAX_PRICE_SERIES_3070="" MAX_PRICE_SERIES_3080="" MAX_PRICE_SERIES_3090="" MAX_PRICE_SERIES_CORSAIR_SF="" MAX_PRICE_SERIES_RX6800="" MAX_PRICE_SERIES_RX6800XT="" MAX_PRICE_SERIES_RX6900XT="" MAX_PRICE_SERIES_RYZEN5600="" MAX_PRICE_SERIES_RYZEN5800="" MAX_PRICE_SERIES_RYZEN5900="" MAX_PRICE_SERIES_RYZEN5950="" MAX_PRICE_SERIES_SONYPS5C="" MAX_PRICE_SERIES_SONYPS5DE="" MAX_PRICE_SERIES_TEST=""

Therefore, it assumes there is no max price, and gives every item which is IN STOCK a mark that its available under buyprice.

Sorry, I overcleaned my dotenv file, it's fixed now. I have price filters, because under thousand price tag (i.e. no separators) works perfectly.

liam-maps commented 3 years ago

i let the script run for wipoid and vsgamers:

Wipoids Config seems most likely out of date, since many cards that are giving responses on the site are not directly listed anymore, however, no card that is configured is reported as instock and > 1000 € ... therefore i was not able to check yet the pricing feature.

checking against vsgamers brought another issue up: vsgamers does not check for max price at all, therefore anything that is available, will get reported as instock and put out by the bot via opening up the page.

For the vsgamers one, I already put a patch up for jef to add. However ... the wipoid one seems to take a bit more effort, since this will take a bit mor time to fix.

wipoid file has a lot of SKUs but some of them are removed when they aren't in stock. They are also removing the "buy now" / "notify me" button placeholder. Neither stores are top tier here, but we must scrape the bottom of the barrel to get a chance.

wofnull commented 3 years ago

so i added a big fix package for wipoid to jefs todo list, depending on how fast he is with adding this in, it will be in main branch shortly.

it will not yet include a bugfix for the maxPrice issue as the site does not behave as it should. jef should take a look into this, because i do not know if the script is currently capable of finding the correct price if its > 1000 €