Ingest all RSS feeds for SAGE - Githubissues

usc-isi-i2 / dig-etl-engine

Download DIG to run on your laptop or server.

http://usc-isi-i2.github.io/dig/

MIT License

101 stars 39 forks source link

Ingest all RSS feeds for SAGE #162

Closed szeke closed 6 years ago

szeke commented 6 years ago

The list of feeds is in https://github.com/usc-isi-i2/sage-research-tool/blob/master/data-sources/SAGE%20RSS%20Feeds%20-%20Sheet1.csv

For the very initial version do the following:

[ ] Run readability
[ ] Have a title
[ ] Run location extraction so we get city, state and country
[ ] Run posting_date extractor

We will add the standard NET SpaCy extractors in the second version, as we need to figure out the ontology for storing the data.