search

ashleychampagne / Web-Scraping-Toolkit

This is the web scraping toolkit for the Brown University Library
11 stars 5 forks source link

Twint Workflow #2

Open ashleychampagne opened 4 years ago

ashleychampagne commented 4 years ago

Fantastic, easy to use workflow. A few comments:

1) The first search (to collect Elon Musk tweets) outputs so many tweets. This is too much for the average user following the tutorial who will need to know how to stop the terminal. Either provide guidelines for how to interrupt the terminal and add a warning about what the command produces or pick a different user name.

2) It would be good if you specifically showed how to gather hashtags

3) It would be helpful to add an output option to the tutorial. Try writing the data to a csv that has different cells that include: a hashtag (e.g. "intersexstories"), user ids, stats (retweets, etc.).

Otherwise, great! Good work.

aidancm commented 4 years ago

Your first two points make sense, and I'll address them on Thursday. As for the third point, I'm not sure how that is different from the .csv instructions under "saving the results." Perhaps what this gets at is that I need to be more explicit about what using --csv will produce? Please let me know what you mean and what I should change with regard to your third point.

ashleychampagne commented 4 years ago

Thanks! I missed your last output section. Sorry about that.

aidancm commented 4 years ago

Addressed these, let me know if it looks good and if there's anything else.