search

EricJMarti / inventory-hunter

⚡️ Get notified as soon as your next CPU, GPU, or game console is in stock
MIT License
1.12k stars 263 forks source link

Microcenter Scraper #66

Closed Remanent closed 3 years ago

Remanent commented 3 years ago

The microcenter scraper doesn't detect items that aren't able to be reserved online, but do in fact have in store inventory.

Example is here

https://www.microcenter.com/product/630284/amd-ryzen-7-5800x-vermeer-38ghz-8-core-am4-boxed-processor?storeid=095

Lists 13 in stock as of 7:30pm EST scraper shows out of stock.

Log from starting a test scraper using above link

D2020-12-05 00:31:20,918 starting with args: /src/run.py --alerter email --email (removed --relay 127.0.0.1 D2020-12-05 00:31:21,150 using parser: lxml D2020-12-05 00:31:21,151 registering custom scraper for domain: amazon D2020-12-05 00:31:21,151 registering custom scraper for domain: bestbuy D2020-12-05 00:31:21,152 registering custom scraper for domain: bhphotovideo D2020-12-05 00:31:21,153 registering custom scraper for domain: microcenter D2020-12-05 00:31:21,154 registering custom scraper for domain: newegg D2020-12-05 00:31:21,155 registering custom scraper for domain: walmart I2020-12-05 00:31:21,168 scraper initialized for https://www.microcenter.com/product/630284/amd-ryzen-7-5800x-vermeer-38ghz-8-core-am4-boxed-processor?storeid=095 I2020-12-05 00:31:31,621 amd-ryzen-7-5800x-vermeer-38ghz-8-core-am4-boxed-processor: not in stock

(time stamp is 5 hours ahead of system time, gmt i suppose)

Here's a link to a store with no in store inventory:

https://www.microcenter.com/product/630284/amd-ryzen-7-5800x-vermeer-38ghz-8-core-am4-boxed-processor?storeid=081

EricJMarti commented 3 years ago

Ah that's a tough one, but I'm up for the challenge! The Micro Center near me is sold out of the 5800X, so the storeid in the URL is critical here. Let's see if I can implement this before it sells out!

Remanent commented 3 years ago

Yeah. hence why i provided the storeid part of the url, it's definitely critical, i checked almost all the stores before i found one with something in stock ;)

Remanent commented 3 years ago

I'll also mention any 3k series gpus at microcenter have been listed the same way, no option to buy them online and pickup in store (no online reservation. i imagine any of the new amd 6k gpu's will listed the same way.

EricJMarti commented 3 years ago

Fixed: https://github.com/EricJMarti/inventory-hunter/commit/7d9ce7c366d7b55a4b1a17075b95f564c6af300f

Please pull the latest image using:

$ docker pull ericjmarti/inventory-hunter:latest
Remanent commented 3 years ago

Unfortunately, not working as intended :( site shows 2 in stock atm, scraper says not in stock.

D2020-12-05 19:33:07,993 [root] starting v1.0.1 with args: /src/run.py --alerter email --email (removed) --relay 127.0.0.1 D2020-12-05 19:33:08,222 [root] using parser: lxml D2020-12-05 19:33:08,222 [root] registering custom scraper for domain: amazon D2020-12-05 19:33:08,223 [root] registering custom scraper for domain: bestbuy D2020-12-05 19:33:08,223 [root] registering custom scraper for domain: bhphotovideo D2020-12-05 19:33:08,224 [root] registering custom scraper for domain: microcenter D2020-12-05 19:33:08,225 [root] registering custom scraper for domain: newegg D2020-12-05 19:33:08,226 [root] registering custom scraper for domain: walmart I2020-12-05 19:33:08,239 [mcrcntr_1] scraper initialized for https://www.microcenter.com/product/630284/amd-ryzen-7-5800x-vermeer-38ghz-8-core-am4-boxed-processor?storeid=101... I2020-12-05 19:33:18,845 [mcrcntr_1] not in stock I2020-12-05 19:33:29,263 [mcrcntr_1] not in stock

EricJMarti commented 3 years ago

Strange, this is what I'm getting:

D2020-12-05 23:37:29,487 [root] starting v1.0.1 with args: /src/run.py --alerter discord --webhook https://discord.com/api/webhooks/...
D2020-12-05 23:37:29,759 [root] using parser: lxml
D2020-12-05 23:37:29,759 [root] registering custom scraper for domain: amazon
D2020-12-05 23:37:29,760 [root] registering custom scraper for domain: bestbuy
D2020-12-05 23:37:29,761 [root] registering custom scraper for domain: bhphotovideo
D2020-12-05 23:37:29,762 [root] registering custom scraper for domain: microcenter
D2020-12-05 23:37:29,763 [root] registering custom scraper for domain: newegg
D2020-12-05 23:37:29,765 [root] registering custom scraper for domain: walmart
I2020-12-05 23:37:29,784 [mcrcntr_1] scraper initialized for https://www.microcenter.com/product/630284/amd-ryzen-7-5800x-vermeer-38ghz-8-core-am4-boxed-processor?storeid=101
I2020-12-05 23:37:31,264 [mcrcntr_1] now in stock at 449.99!
Remanent commented 3 years ago

I’ll have to try again in the morning, I’m won’t be able to test it until then..

I had don’t both a git pull, and a pull dicker to get the latest.. is there anything else I need to do or delete first? I’d killed and pruned the running containers before trying to test the latest container you had posted.

On Dec 5, 2020, at 6:59 PM, Eric Marti notifications@github.com wrote:



Strange, this is what I'm getting:

D2020-12-05 23:37:29,487 [root] starting v1.0.1 with args: /src/run.py --alerter discord --webhook https://discord.com/api/webhooks/... D2020-12-05 23:37:29,759 [root] using parser: lxml D2020-12-05 23:37:29,759 [root] registering custom scraper for domain: amazon D2020-12-05 23:37:29,760 [root] registering custom scraper for domain: bestbuy D2020-12-05 23:37:29,761 [root] registering custom scraper for domain: bhphotovideo D2020-12-05 23:37:29,762 [root] registering custom scraper for domain: microcenter D2020-12-05 23:37:29,763 [root] registering custom scraper for domain: newegg D2020-12-05 23:37:29,765 [root] registering custom scraper for domain: walmart I2020-12-05 23:37:29,784 [mcrcntr_1] scraper initialized for https://www.microcenter.com/product/630284/amd-ryzen-7-5800x-vermeer-38ghz-8-core-am4-boxed-processor?storeid=101 I2020-12-05 23:37:31,264 [mcrcntr_1] now in stock at 449.99!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/EricJMarti/inventory-hunter/issues/66#issuecomment-739432959, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAKQ67Q4KH6X7WZ2N7GUDS3STLCPNANCNFSM4UOEEOXQ.

Remanent commented 3 years ago

bah, I had an error in my yaml file. somehow the … ended up on the same line as the URL, and thus the file was probably being incorrectly read. once I placed it back as the last line, it worked properly. Thank you for your time.