AdamGagorik / ffxiahbot

A python module for interacting with a darkstar server.
MIT License
22 stars 57 forks source link

Default item.csv stops at 19803, scrubbing won't go above that either #42

Open LaernuTairos opened 2 years ago

LaernuTairos commented 2 years ago

Hey Adam.

I'm aware of the warning but I noticed the itemid list during scrubs won't pull item past adaman bullets 19803. The default list only goes up to that as well. But looking at the FFXIAH site and the SQLs for LSB, this should be going up to almost 30000. So items are missing it looks like. Example: 28314 Aetosaur gear, can't get it in the CSV, unless we manually do that, but so much is missing it would take soooo long lol. It seems like this may be on purpose because alot of items right after those bullets are rare/ex but those stop eventually & the IDs goes back to sellable items

Is there a command we can use to force the scrubber to go up to that #? Or is it already supposed to do that and I'm messing something up here. Maybe I got an old item.csv somehow, or it's an issue? Thanks!

ParalHAXX commented 11 months ago

I have the same problem, i would like to pull prices for all the new items, but doesn't know how to do it without manually adding them one by one into the .csv, which is somewhat really painful.

If someone have an idea, i will be really interested.

Thank you

AdamGagorik commented 2 months ago

The download logic for the scrubber works by using category URLs.

For example, they look like this: http://www.ffxiah.com/browse/62/grips.

It seems like I stupidly hard-coded the max url integer to be 240. I have no idea why.

AdamGagorik commented 2 months ago

Here is the line that limits the category URL: https://github.com/AdamGagorik/ffxiahbot/blob/a3e77108936af4b4b70e7dfc96d769faba666444/ffxiahbot/scrubbing/ffxiah.py#L81

EDGECOM666 commented 1 month ago

check this out AdamGagorik

https://github.com/EDGECOM666/FFXIAHBOT_CSV

if you want to use my items.csv feel free