Closed kienmarkdo closed 7 months ago
python scrape.py
--max-entities <number_of_entities> # number of most recent entities to be collected in this collection
--throttle-time <seconds> # override default throttle time between API calls
--get-messages <number_of_msgs> # max. number of messages to collect per entity
--get-participants
--get-entities
--export-to-es # export results to Elasticsearch
Use argparse to allow the user to specify collection details in the command line.
Example
Allow to specify limits to collection. Such as 1000 messages, 10 minutes of elapsed time, etc.