Create mailinglist mirror collection based on https://github.com/datactive/bigbang/issues/446

datactive / bigbang

Scientific analysis of collaborative communities

http://datactive.github.io/bigbang/

MIT License

149 stars 52 forks source link

Create mailinglist mirror collection based on https://github.com/datactive/bigbang/issues/446 #447

Closed nllz closed 2 years ago

nllz commented 3 years ago

Create a github-lfs with all mailinglist data
Ideally scraping all lists once per week
Use this archive a pointer in notebooks? (will minimize downloads + serve the online/binder instance)

Christovis commented 3 years ago

the github-lfs as it needs to be payed for starting from 1GB

nllz commented 3 years ago

Let's use the server I provided.

Christovis commented 2 years ago

Using Niels server for now and waiting for Covington advice

nllz commented 2 years ago

Awesome! I have a couple of q's: Do you have rsync working with weekly updates as well? Does it work for ICANN, RIPE, IETF, and 3GPP ? Does it get the mailinglists from the GitHub files?