gwu-libraries / TweetSets

Service for creating Twitter datasets for research and archiving.
MIT License
25 stars 2 forks source link

Refactor tweetset_loader.py for greater flexibility #134

Open dolsysmith opened 3 years ago

dolsysmith commented 3 years ago
lwrubel commented 2 years ago

Add option to specify concatenating JSON files up to a size.

Also move parse_size() from spark_utils.py to utils.py since it will need to be called to supply to_size value.