Build mirrors network

[veganstraightedge]

Depends on #1823

---

Right now, crimethinc.com runs as a Rails app on Heroku hosted in the U.S.

If any actor decides to target, take down, or block the site, we'd like to be able to point DNS to a different server and not miss a step (or too many steps). That means, we'd need to have the site already up and running in different contexts:

• different companies
• non-company backed
• different countries

These mirrors need to remain unknown to the general population and remain behind the scenes if they're ever needed.

So, a couple key ingredients to making this work seamlessly are:

• the mirrors need to update themselves when new code is pushed to main branch in GitHub (or a GitHub mirror)
• the mirror need to update their database content (everything except drafts and users) like we do for development

Read replica mirrors

These could be run by anyone (in theory) but not update their content directly. Only get their content updates from the .com. Their database content would like the database dump used for development staging, and review apps.

Live replica mirrors

These would be run by us or people we trust completely. They would need all fo the public data plus drafts, users, article view counts. And their content could be updated directly if we made a DNS change to point to one of them.

[just1602]

It'd be possible, but it'd be a lot of job for us to do. An easier way would be to dockerize the present application and create another docker image every 24h or something like that which contains a static version of the site. So if we get ban, we just pop up a new static site in a second or a couple of minutes since it's gonna contains everything we have including audio, videos, images, etc.

Then once the static version is setup we can take the time we need to setup a new instance of the CMS.

Another thing we should think about is that if we get massively ban, it could be quick for us to lose everything even if we've backup since we only rely on propitiatory solution. What I mean by that is if in less than a couple of hours we lose heroku, our backups, github / microsoft, docker hub ans stuff even if we have dockerize the application we should think about a way to get some infrastructure running that we manage and we have control over.

[goncalopereira]

:+1: on getting the static version up first. Looking for graceful degradation, in this case into read only.

[reedjones]

Does a static version of the site already exist somewhere?

[goncalopereira]

@ramonabrajdic hi no, that was a proof of concept I didn't finish it, I did take some notes on things that were modified whilst scrapping the website and generating a static copy.

[reedjones]

in the setup script /script/seed, does that download the complete current data or just like a few samples? Would it make since to have like a public backup of the database?

then from that can use a static site generator like jekyll or something to generate mirrors

so basically could have a public script to generate a static site, and public backups of the database, so at anytime someone can get the backup data and use the site generator to make a mirror

[goncalopereira]

@veganstraightedge will know best but I believe that is just test data for development and the production data is not available.

[veganstraightedge]

@reedjones @goncalopereira The script/seed does indeed download a full production dump of published data. So, no drafts of articles, zines, etc. But that's ok.

I like the idea of creating a static site from the published database dump! 👍🏻

An interesting challenge will be how to keep the static site's template in sync with changes in the production site's templates.

[reedjones]

@veganstraightedge maybe something like this: https://docs.ipfs.io/how-to/host-git-style-repo/
like whenever a change is made to the database of articles for example, it pushes to ifps, or a torrent, or something like this, and then basically we could have a separate, script/seed-from-peers that gets the data from some p2p network?

so each instance of the site would also keep a list of peers, so if for example the main site was taken down, there would be a bunch of peers that have list of peers, and so on, that could load all the data and serve it either locally or over the internet, or however. ala scuttlebutt https://scuttlebutt.nz/ (for example)