search

w3c / modern-tooling

Work of the modern tooling task force
http://w3c.github.io/modern-tooling/
MIT License
44 stars 39 forks source link

Automatic snapshot generator #75

Open plehegar opened 6 years ago

plehegar commented 6 years ago

Idea here is to have the ability to generate snapshots based on git commits.

xfq commented 6 years ago

Currently, we can already link to snapshots with RawGit/HTMLPreview (without the WHATWG warning, tho), because they support using a specific tag or commit hash in the URL.

However, one issue with RawGit/HTMLPreview is, if the spec uses Bikeshed and auto-deploys the generated index.html to gh-pages with Travis CI, you need to find the corresponding commit in gh-pages, rather than using the commit in master directly.

Our friends at WHATWG do this in build-time: https://github.com/whatwg/whatwg.org/blob/e8cda18f1f9168b525f9612ce4ca49d7a92f6430/resources.whatwg.org/build/deploy.sh#L93-L103

They also include a client-side script in the boilerplate, letting the spec reader go to the commit snapshot by pressing y (similar to GitHub). (Bikeshed PR: https://github.com/tabatkins/bikeshed/pull/288)

plehegar commented 6 years ago

Here is what I think we'd need: provide a link to the commit snapshot, provide a history of snapshots, works with respec and bikeshed, easily deployable.

xfq commented 6 years ago

If we don't want to use RawGit/HTMLPreview, should we save the commit snapshots on GitHub (not in the default branch)? If so, we will have 1,000 separate files if there are 1,000 commits (and that doesn't even count the multiple-file spec case). PR Preview stores the specs on AWS, but AWS is not easy to access in China...

xfq commented 6 years ago

I just created a repo for this: https://github.com/xfq/commit-snapshot-tool

plehegar commented 6 years ago

if we need a cloud server, maybe we can use a labs.w3.org?

tabatkins commented 6 years ago

Note that pr-preview already exists and is almost this functionality; it works with Bikeshed and ReSpec and generates previews for any PR.

xfq commented 6 years ago

Note that pr-preview already exists and is almost this functionality; it works with Bikeshed and ReSpec and generates previews for any PR.

Yep. It uses a JSON file to reflect the configuration of the spec, and uses the Bikeshed CGI and Spec Generator with raw links on GitHub and RawGit to generate the spec, and saves the generated spec on AWS.

xfq commented 6 years ago

And drafts.csswg.org also has this functionality, e.g., https://drafts.csswg.org/history/css-values-4/

(Source)

xfq commented 6 years ago

Maybe we can reuse some code in the projects above, and deploy it to labs.w3.org.

xfq commented 6 years ago

@tobie Could you provide some hint on how we should build this tool (or even better, we can collaborate on extending pr-preview)?

I think we should monitor the push event, detect the config of the repo (either by reusing .pr-preview.json, or by creating a new config file), build the spec using Bikeshed/ReSpec's web service, and save the generated snapshot on some server. We probably also need frontend code to display a history of snapshots of a given spec.

tobie commented 6 years ago

I agree you can probably reuse large chunks of pr-preview for this. And the overall architecture you describe looks good to me.

I'd be happy to help, but that would require setting up some consultancy agreement which I doubt W3C has funds for.

PR Preview stores the specs on AWS, but AWS is not easy to access in China...

Oh! That's interesting. Is the filtering happening on the domain name or on the actual IP? If it's the former, adding custom domain names is easy. That's what we've done for WHATWG, e.g.: https://whatpr.org/html/3714/iframe-embed-object.html is hosted on AWS S3.

xfq commented 6 years ago

PR Preview stores the specs on AWS, but AWS is not easy to access in China...

Oh! That's interesting. Is the filtering happening on the domain name or on the actual IP? If it's the former, adding custom domain names is easy. That's what we've done for WHATWG, e.g.: https://whatpr.org/html/3714/iframe-embed-object.html is hosted on AWS S3.

It's a sophisticated system, consisting of DNS filtering, IP blocking, TCP ports blocking, and other blocking methods. Some (but not all) AWS servers are not accessible in some regions/ISPs in China, and the list of servers being blocked is constantly changing.

For example, in my ISP: