search

Spaceface16518 / strat-collector

Combs through a specific subreddit for posts with over a certain threshold in comments. It then adds them to a database.
MIT License
2 stars 0 forks source link

Choose what data to extract #2

Open Spaceface16518 opened 6 years ago

Spaceface16518 commented 6 years ago

The streamPost gets tons of data about posts, but what data gets extrapolated needs to be decided. This depends on the future vision for this project, which is in @vorana's wheelhouse.

Spaceface16518 commented 6 years ago

As of commit a7e96ba, the data that is being extracted is just:

vorana commented 6 years ago

we also need to extract:

Spaceface16518 commented 6 years ago

@vorana I am working on all of those except for

Gilded

I have not yet figured out the correct key for it, but as soon as I test it I will find out. The changes will most likely be added to the export branch.

EDIT: I also am adding a netUps property to data for easy access.

Spaceface16518 commented 6 years ago

Because of mongo, we don't really have to extract any data. We can just send the data straight from reddit into the database.

vorana commented 6 years ago

No, remember we want to format the data before we put it in that database. Will make it much easier later.