Updating website in github publishes changes to IPFS && to ipfs.io

[harlantwood]

Edit: Original title of this issue was: Idea: Publishing website to IPFS auto-publishes to github pages

Background: I've been having fun setting up continuous deployment to github pages recently for a few of my own sites. For example, when anyone updates the site source on github for dstack.org, CI notices the changes, builds the site from source, and pushes the resulting pages to gh-pages.

We could certainly do the same or similar for ipfs.io. But maybe we could do something much cooler, which could serve as an example to boot.

The basic flow I have in mind is:

• publish the ipfs.io site at an ipns address
• watch that ipns address for changes (maybe now using a cronjob, later using a subscribe mechanism -- if I understand the current state of things)
• when there are changes, fetch the latest code from IPFS, and push it to github pages

What would all this get us? A fantastic example of publishing content to the "permanent web" first, then mirroring it to github pages, a lightning fast, highly scalable free hosting system, which also hooks into DNS with custom domain names. That is to say, it's a bridge, to allow pioneers who are publishing on IPFS to mirror their work to the "old web".

Caveat: I'm still an IPFS noob, tho enthusiastic. If this sounds interesting, I'd be happy to throw a bit of development at it. Feedback and upgrades welcome.

[harlantwood]

CC @jacksenechal

[harlantwood]

@jbenet interested to hear your thoughts on this when you have a chance...

[jbenet]

Hey @harlantwood finally getting around to this one.

This would definitely work, but can skip the gh-pages entirely (if there is someone willing to provide the data -- we're considering this sort of thing). Can skip gh-pages because IPFS supports a way to serve arbitrary websites at their own domains, with two DNS records: a TXT and an A.

TXT dnslink=/ipfs/path/to/content
A <ip-address-of-gateway>  # this probably should be a CNAME

I'd love for someone to write up an example of how to do this (maybe @Luzifer or @kevinwallace?) or maybe adding it to https://github.com/ipfs/examples/tree/master/examples/websites (@whyrusleeping).

[RichardLitt]

I think this is pretty much done with the last merge in #59, which automatically sets the DNS record.

[jbenet]

yeah last part would be to automate the publish with a githook webhook or something

[RichardLitt]

Can't do that without publicly providing digitalocean keys.

[jbenet]

well, would have a githook to a machine we control with the token or something

[harlantwood]

Maybe we can provide the DO auth token as an env variable in Travis or Circle and publish from there?

73

[jbenet]

yeah maybe, though not sure if those creds might be leakable through their public webuis.

we could write a nice dedicated webhook bot for people to deploy on heroku or something. maybe this is something https://github.com/diasdavid/ipscend could help with-- particularly if it did the dnslink-deploy part too-- then the hook would be very simple.

the hook could still do it, but would be more complicated, would need to:

• use ipscend (or have to run some build process, then ipfs add -r on a configured dir.)
• get the output hash
• run dnslink-deploy on a configured dns domain with the output hash.

[daviddias]

Doesn't gh-pages require that a repo with a gh-pages branch to exist? We could add an option on ipscend for 'onpublish' to also push to the gh-pages branch, would that fulfil the expectation?

[RichardLitt]

We could just make gh-pages the main branch. GitHub allows that option, and we don't need master to be the trunk.

[harlantwood]

I think this thread is no longer about github pages. I'll change the title to reflect that. It's about deploying to IPFS triggered by a git commit, ie from a github webhook.

I've been playing with this for IPFS/ci-status#1, and found a nice module to do this with node... Something like:

githubhook = require 'githubhook'
github = githubhook
  port: process.env.PORT
  path: '/'
  # secret: 'xxx'

github.listen()
github.on '*', (event, repo, ref, data) ->
  # build, deploy, whatever...

I plan to build this anyway to deploy the CI dashboard on any github commits to its own codebase; we could certainly repurpose the same idea (or even the same server instance) for building and deploying the website.

Questions:

1. I am prototyping this on heroku. Should we plan to run there, or elsewhere, eg digital ocean? Heroku is easy, and a free instance should work fine, but it's also nice to have all our stuff in one place... My recommendation: heroku
2. Should we have a single codebase to manage any git hooks, which would build/deploy etc as needed? Or split it into CI / website / etc codebase / servers? My recommendation: single codebase, at new repo: ipfs/githooks

[RichardLitt]

I'm always for putting everything in one place: I would vote for using DigitalOcean, and for using ipfs/githooks.

Everything else looks great.

[harlantwood]

Sounds good, will prototype on my own server and ping when it's baked.

[jbenet]

@harlantwood im ok with heroku for things like these -- particularly since it's free :). the tricky bit is accounting all apps hosted there and so on. i could get us an account etc.

[jbenet]

also ok with one or several codebases (if several have one be an index)

[harlantwood]

I'll keep prototyping on heroku for now... not sure if the free tier will work, or the $7/mo hobby tier would be needed.

Going with one codebase, at least for now.

[harlantwood]

This development is moving forward in https://github.com/ipfs/infrastructure/issues/137; closing this issue.