Where are you scraping data from?

akiokaki / SI507_final_project

SI507 Final Project.

0 stars 1 forks source link

Where are you scraping data from? #1

Open akiokaki opened 6 years ago

akiokaki commented 6 years ago

Preliminary ideas: -Rakuten to search for a keyword, generate a list of products, and scrape prices.

Tumblr data to create a tab of what sort of posts are popular

anandpdoshi commented 6 years ago

Details, details! What kind of data will you get from scraping, what tables will you create and store this data in, and how will you visualize it?

akiokaki commented 6 years ago

OK new idea: 1) Request keyword from user. 2) Search keyword on Amazon and Rakuten 3) Scrape the first upto 10 searches and extract the prices for each search 4) Throw the values into its own database, Find an average 5) Plot a bar graph for both websites...if possible a box and whiskers plot

=== If I have time 6) See if price dropped previously. 7) Send text to phone if price drops from previous time scraped data, using Twilio