Address issue 460 - Githubissues

datactive / bigbang

Scientific analysis of collaborative communities

http://datactive.github.io/bigbang/

MIT License

149 stars 52 forks source link

Closed Christovis closed 3 years ago

Christovis commented 3 years ago

This PR addresses issue #460.

Implements the following:

changes the logger filemode from appending ('a') to erasure of previous content and rewriting it ('w')
use pathlib to take care of folder structure instead of os.mkdir

use tqcdm for visual feedback on messages scraping progress:

❯ python bin/collect_mail.py -f ../listserv.test.txt     
3GPP_TSG_RAN_ADHOC: 100%|##################################| 69/69 [02:04<00:00,  1.80s/it]

Christovis commented 3 years ago

As no other modules but the listserv one is affected and all test pass I merge it.