driscoll42 / ebayMarketAnalyzer

Scrape all eBay sold listings to determine average/median pricing, plot listings over time with trend lines, and extract to excel
195 stars 26 forks source link

Use the min_date functionality to improve the binary search #30

Closed driscoll42 closed 3 years ago

driscoll42 commented 3 years ago

Is your feature request related to a problem? Please describe. Right now when a search is done, the code keeps splitting an ebay search into chunks until they are 800 items or fewer. This is because eBay only lets you go back 800 items at a time. To get around this the code changes the min/max price until there are 800 items or fewer. This can result in inefficiencies as it may scan over already scarped items.

Describe the solution you'd like Since the items are sorted by date ended, if we use the min_date parameter and the most recent sold date already scraped, we can split if there are fewer than 800 items, or on page 4 of the ebay search the min date is >= most recent sold date - min_Date