search

enzoampil / fastquant

fastquant — Backtest and optimize your ML trading strategies with only 3 lines of code!
MIT License
1.54k stars 239 forks source link

News article scraper #6

Open jpdeleon opened 4 years ago

jpdeleon commented 4 years ago

Newspaper seems like a good tool to scrape and curate articles related to PSE-listed stocks.

I can imagine using a different tool to search for recent news related to a company and using newspaper to scrape that article.

What do you think?

Edit:

enzoampil commented 4 years ago

@jpdeleon I think this makes sense to do - was intending to do something similar with Twitter feeds. However, do take note it will only add value as a data source if we create a pipeline that pulls new PSE related article URLs, perform scraping on those URLs, and then store the data on a separate DB. I do agree that integrating the functions to do this into psequant would make sense.

Glad to have further discussions on how to implement this, specifically on how to get PSE related article URLs so that we can use newspaper to do the actual scraping :)

jpdeleon commented 4 years ago

@rafmacalaba I added a link above about existing news scrapers. Please let me know which is best for our purpose.

rafmacalaba commented 4 years ago

@jpdeleon newspaper looks promising!