Updated models & tests - Githubissues

gwu-libraries / TweetSets

Service for creating Twitter datasets for research and archiving.

MIT License

25 stars 2 forks source link

Updated models & tests #155

Closed dolsysmith closed 2 years ago

dolsysmith commented 2 years ago

to_tweet function has been updated (and its dependent functions) to align it with twarc json2csv 1.12.1. This affects the following:

text in quoted_status is NOT added to the text field.
hashtags from retweeted_status.extended_tweet.entities and retweeted_status.entities are included.
urls are included from both extended_tweet.entities as well as entities (as these can be different).
favorite_count is included from retweeted_status (when present).

Testing

Load a collection with this branch to compare with the same collection loaded with the master branch.

Compare results for all types for keyword and hashtag searches.
Counts will differ, especially for quote type, though the changes above will affect the other counts to some degree, too.

lwrubel commented 2 years ago

Tested and looks good to merge.