search

w3c / web-roadmaps

Framework for Web technology roadmaps
190 stars 39 forks source link

readme

Framework for Web technology roadmaps

This repository hosts a framework to present roadmaps of ongoing and future Web technologies.

It aims at simplifying the creation and maintenance of such roadmaps by collecting automatically information about standardization and implementation status of features described in W3C specifications and others.

Available roadmaps

Table of contents

Overview of the framework

A roadmap is defined here as a collection of features that make up a consistent set of technologies used to build a particular type of applications (e.g. media applications, games) or to promote a particular aspect of application development (e.g. security). Each feature is described in prose to explain its relevance to the theme of the roadmap. From that prose, the framework generates tables (one per section) summarizing the standardization and implementation status of the specification(s) that define(s) the said feature.

A given roadmap is expected to contain one or more of the following sections describing the high-level status of the features described in the section:

Multiple roadmaps can be associated in a single multi-page roadmap with a front index page that links to individual roadmap pages. The framework generates a navigation menu in each individual roadmap page to navigate between them.

In short, the goal of the framework is to allow roadmap authors to focus on the prose that describes features that compose the roadmap, the framework taking care of adding implementation data for each feature that composes the roadmap and of formatting the result, including providing means for users to navigate between pages.

While the pages are generated dynamically in the browser, it is likely preferable to publish exported versions of the roadmaps since the framework has not been optimized for performance.

Adding a feature to a roadmap

A feature is roughly speaking a piece of technology that the target audience of the document would recognize as something they need to build the product they are interested in.

For the 3 first categories of features described above, a feature comes with one or more specifications that covers it. The process to add a feature to a roadmap or add a specification to an existing feature is as follows:

If you want to reference a specification's feature but not have it appear in summary tables at the end of each section, add a data-linkonly attribute (set to whatever value). For instance <a data-featureid='webaudio/worklet' data-linkonly>AudioWorklet</a> would link to the AudioWorklet definition in the Web Audio API but would not list this entry in the following table. This can be useful to reference specs that can be used in conjunction with the feature being described.

JSON format for describing specifications

Each specification is described by a JSON object that will allow retrieving information about the standardization status of the spec and its level of implementation in browsers. Note the framework will automatically generate an empty data file when it does not exist for W3C specs (see Generate content locally for details), allowing authors to reference a W3C spec to start with without having to worry about creating the data file.

That JSON object is stored in a file in the data directory, whose name is then used to refer to the said specification from relevant features.

Depending on the advancement of the underlying specification, the JSON object can have the following properties:

Here is an example of a JSON file that describes the "Intersection Observer" specification:

{
  "url": "https://www.w3.org/TR/intersection-observer/",
  "impl": {
    "caniuse": "intersectionobserver",
    "chromestatus": 5695342691483648,
    "webkitstatus": "specification-intersection-observer",
    "edgestatus": "Intersection Observer",
    "mdn": "api.IntersectionObserver"
  },
  "polyfills": [
    {
      "label": "Polyfill.io",
      "url": "https://polyfill.io/v2/docs/features/#IntersectionObserver"
    }
  ]
}

Describing implementation status

Note (March 2018): implementation status features are still being worked upon in the framework. The implementation description should remain backward compatible, but it may still evolve.

Provided the description contains relevant information, the framework can automatically retrieve the implementation status in main browsers from the following Web platform status sources: Can I use, Chrome Platform Status, MDN Browser Compatibility Data, Microsoft Edge web platform features status and roadmap and WebKit Feature Status.

To enable this, the decription must contain an impl property whose value is an object with one or more of the following optional properties:

From time to time, platform status sources may:

When this happens, you may use the other sub-property to specify implementation status manually. Property value must be an array of objects that have the following properties:

For instance, let's say that "Can I use" report that a particular feature is in development in WebKit, whereas you know that the feature has not yet been considered there; and that it does not report anything on status in Edge, whereas you know from discussion with the Edge team that it is being considered, you could add:

{
  "TR": "...",
  "impl": {
    "caniuse": "...",
    "other": [
      {
        "ua": "webkit",
        "status": "",
        "source": "feedback",
        "date": "2018-03-19",
        "comment": "No public signal in WebKit for the feature, information reported by Can I Use is incorrect"
      },
      {
        "ua": "edge",
        "status": "consideration",
        "source": "insight",
        "date": "2018-03-19",
        "comment": "From discussion with the Edge team, the feature is under consideration"
      }
    ]
  }
}

In the previous example, the information on webkit will override the information reported by "Can I Use", whereas the information on edge will not be used if "Can I Use" asserts that the development has started or that the feature has shipped in Edge.

Important: As noted above, maintaining implementation information over time is difficult and error prone. Whenever possible, the implementation status of a particular feature should be automatically extracted from main sources, and the other mechanism should only be used as a fallback when implementation status is incorrect or not available.

Creating a new roadmap page or a new single-page roadmap

Start from the following template

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Title of the roadmap</title>
  </head>
  <body>
    <header>
      <h1>Title of the roadmap</h1>
      <p>Description of the scope of the roadmap and to whom it matters</p>
    </header>
    <main>
      <section class="featureset well-deployed">
        <h2>Well-deployed technologies</h2>
      </section>
      <section class="featureset in-progress">
        <h2>Specifications in progress</h2>
      </section>
      <section class="featureset exploratory-work">
        <h2>Exploratory work</h2>
      </section>
      <section class="not-covered">
        <h2>Features not covered by ongoing work</h2>
        <dl>
          <dt></dt>
          <dd></dd>
        </dl>
      </section>
      <section class="discontinued">
        <h2>Discontinued features</h2>
        <dl>
          <dt></dt>
          <dd></dd>
        </dl>
      </section>
    </main>
    <script src="https://github.com/w3c/web-roadmaps/raw/main/./js/generate.js"></script>
  </body>
</html>

If adding to an existing roadmap, you should also edit the toc.json file to add a link to that new page.

For a new single page roadmap, you also need to create a toc.json file as described below.

Creating the index of a new multi-page roadmap

If you want to divide your roadmap as a multipage document, you also need to create an index page and a JSON file listing the various sub-pages.

The template for the index page is as follows:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Title of the roadmap</title>
  </head>
  <body>
    <header>
      <h1>Title of roadmap</h1>
      <p>Introduction of roadmap scope</p>
    </header>
    <script src="https://github.com/w3c/web-roadmaps/raw/main/./js/generate.js"></script>
  </body>
</html>

The template for the JSON file listing sub-pages is as follows:

{
  "title": "Title of the Roadmap",
  "discourse": {
    "category": "Category of the discourse instance where to post suggestions for new feature (leave empty if none)",
    "url": "https://discourse.wicg.io/"
  },
  "github": "https://github.com/[org]/[repo]",
  "pages": [
    {
      "url": "subpage.html",
      "title": "Title of the subpage",
      "icon": "https://.../icon/to/use/in/the/index/page",
      "description": "One line description of the scope of the subpage"
    }
  ]
}

The following settings may be added to the toc.json file to generate the appropriate Document Metadata section near the top of the index page:

The above settings may also be passed to the page as query string parameters, which can be useful to generate specific snapshots (in particular to pass the publishDate parameter). For instance, supposing the page can be served over a local HTTP server running at port 8080, you could use the following to view a "complete" Document Metadata section:

https://localhost:8080/mobile/?publishDate=2018-04&previousVersion=https://www.w3.org/2018/01/web-roadmaps/mobile/&publishedVersion=https://www.w3.org/Mobile/roadmap/

Note: this last mechanism cannot be used on pages that have already been generated, such as those published under https://w3c.github.io/web-roadmaps/.

Creating an About this document page

In a multi-page roadmap, it is good practice to link to an "About this document" page that provides useful information about the contents and status of the document. The about page is a specific page in that it does not appear in the index page along with the other pages. As such, it needs to be defined in a separate about property in the JSON file that lists sub-pages, as in:

{
  "title": "Title of the Roadmap",
  "pages": [],
  "about": {
    "url": "about.html"
  }
}

The url property is the only mandated property to describe the about page, but note you may also provide a title property to force a different title in the side navigation menu (default is "About this document"), and an icon property to provide a custom icon.

The referenced file must exist. The framework defines a default "About this document" page whose contents you can reference using a data-contents="about" HTML attribute. A minimal about page would be:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>About this document</title>
  </head>
  <body>
    <header>
      <h1>About this document</h1>
    </header>
    <main data-contents="about"></main>
    <script src="https://github.com/w3c/web-roadmaps/raw/main/./js/generate.js"></script>
  </body>
</html>

Children of the <main> element in the about page are automatically appended to the default "About this document" page. You may insert elements before existing ones or replace contents by using a data-insertBefore or data-replace attribute along with a CSS selector. For instance, the following about page would use the default one and replace the first paragraph of the Goals section (which is the second sibling in that section), add an "Authors" section before the "Audience" one, and finish the page with a "Share the love" section:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>About this document</title>
  </head>
  <body>
    <header>
      <h1>About this document</h1>
    </header>
    <main data-contents="about">
      <p data-replace="#goals p:nth-child(2)">
        This document brings the Web to its full potential.
      </p>
      <section data-insertBefore="#audience">
        <h2>Authors</h2>
        <p>I!</p>
      </section>
      <section>
        <h2>Share the love</h2>
        <p>❤</p>
      </section>
    </main>
    <script src="https://github.com/w3c/web-roadmaps/raw/main/./js/generate.js"></script>
  </body>
</html>

Creating feature sections

As explained above, individual features are grouped by an overall feature name described in a data-feature attribute. This grouping is not shown to readers in the prose by default (it merely appears in summary tables). To make pages more readable when the number of individual features they reference grows, roadmap authors may want to create sections for the overall feature names.

The framework will do that automatically if the toc.json file contains a createFeatureSections property set to true.

Customizing summary tables

The framework automatically generates and renders summary tables at the end of sections that are flagged with a featureset class. Summary tables contain one entry per feature mentioned in the prose of the section. The columns rendered in the summary table are also determined by the class attribute of the <section> tag. Recognized values are:

Roadmap authors may customize the columns displayed and create new types of tables through the tables property of the toc.json file. That property must be set to an object indexed by the class identifier that will trigger the use of the table (e.g. well-deployed to override the default definition of the well-deployed table, or a new name to create a new type). For each type, the list of columns to render must be given in an array of objects that describe the column to render.

That object must contain a type property that identifies the type of column. It may also contain a title property to override the default column's title, and other parameters (which depend on the type of column).

The framework recognizes the following column types:

For instance, to add the seeAlso column with all possible links to the summary table rendered at the end of well-deployed sections, and to create a new type of section for reference documents that also renders a seeAlso column with only the Editor's Draft and links flagged as ref in the definition of specifications, you may add the following to your toc.json file:

"tables": {
  "well-deployed": ["feature", "spec", "maturity", "impl", "seeAlso"],
  "reference": ["spec", "maturity", {
    "type": "seeAlso",
    "title": "Reference documents",
    "kinds": ["edDraft", "ref"]
  }]
}

With these definitions, the reference table will be generated at the end of sections that have a class attribute set to featureset reference.

Repository branches

The source of the roadmaps is in the main branch. This is the default branch of the repository, and the branch against which you should be sending pull requests. Whenever content is pushed onto the main branch, a job will run, fetch information and implementation data for all features listed in data, and eventually update the gh-pages branch accordingly.

The gh-pages branch is the branch published on https://w3c.github.io/web-roadmaps/.

Generate content locally

If you would like to visualize the contents of a roadmap locally as it would appear on the published version, you will need to:

  1. Create a W3C account and a W3C API key if not already done.
  2. Create a config.json file in the root of the repository that contains a w3cApiKey property with a valid W3C API key.
  3. Run npm run all to update information and implementation data. This should generate .out/data/tr.json and .out/data/impl.json files. It should also validate the data files, the generated files and the HTML files. Last but not least, it should generate static versions of the roadmap pages in an .out folder. Note you'll need Node.js v8.0.0 or above and you'll need to run npm install first.
  4. Browse the contents of the .out folder on your favorite Web browser (opening the file works fine, no need to serve the file over HTTP).

The npm run all script can take some time. If you want to have a more interactive way to browse updates you're making to files, you may follow these instructions instead:

  1. Create a W3C account and a W3C API key if not already done.
  2. Create a config.json file in the root of the repository that contains a w3cApiKey property with a valid W3C API key.
  3. Run npm run create-missing-data to create missing data files. Only needed if you referenced new specs.
  4. Run npm run generate-info to update information and implementation data. This should generate .out/data/tr.json and .out/data/impl.json files. This step needs to be run again whenever you make changes to information in the data folder.
  5. Serve the root folder over HTTP (any simple HTTP server should work), and browse the roadmap files over HTTP in your favorite Web browser. Refresh the content whenever you've made changes to the HTML, JS, or data files.

Translating a roadmap

Note (January 2018): this part should be mostly stable now, but it may still evolve based on practical experience with translating roadmaps.

The translator needs to provide:

To maintain translations over time, it is wise to use Git tags to create snapshots of a roadmap whenever a significant update is needed. Maintaining the translation then becomes a matter of comparing the latest snapshot with the previous one. The translator can retrieve the ZIP that contains the HTML files to translate from GitHub.

Translating HTML pages

That is the main content that needs to be translated. A few recommendations:

Creating a translations.xx.json file

The js/translations.xx.json file, where xx is the BCP47 language code, needs to contain the translations of all specification titles, group names, feature names and other labels used within the roadmaps that need to be translated. This file is common to all roadmaps. It should respect the following structure:

{
  "sections": {
    "well-deployed": "",
    "in-progress": "",
    "exploratory-work": "",
    "not-covered": "",
    "discontinued": ""
  },
  "columns": {
    "feature": "",
    "spec": "",
    "maturity": "",
    "impl": "",
    "implintents": "",
    "seeAlso": "",
    "milestones": ""
  },
  "implstatus": {
    "shipped": "",
    "experimental": "",
    "indevelopment": "",
    "consideration": ""
  },
  "labels": {
    "N/A": "",
    "%feature in %spec": "",
    "Polyfills": "",
    "Home": ""
  },
  "groups": {
    "CSS Working Group": "",
    "...": ""
  },
  "specifications": {
    "CSS Animations": "",
    "...": ""
  },
  "features": {
    "audio element": "",
    "picture element": "",
    "...": ""
  },
  "maturity": {
    "ED": "",
    "WD": "",
    "LS": "",
    "CR": "",
    "PR": "",
    "REC": "",
    "Retired": "",
    "NOTE": "",
    "REF": ""
  }
}

The translations of section titles (sections), table columns headers (columns), implementation status (implstatus) and labels (labels) are required. Translations of group names, specification titles and feature names are optional, although recommended. The framework will default to English when a translation is missing.

Note the framework will also write the English version of specification titles and of group names next to their translations in the generated tables, because the English version is often used when referring to specs and groups in Web pages, regardless of the language of the page.

Translating the toc.json file

Create a toc.xx.json file that follows the same structure as the toc.json file. The contents of the localized version will overwrite the same content in the toc.json. You do not need to duplicate content that does not need to change. For instance, in the following translation in French of the example JSON file listing sub-pages that appears earlier in this document, note that the URL of the Discourse instance and the URL and icon of the subpage have not been duplicated in the localized file:

{
  "title": "Le titre de la feuille de route",
  "discourse": {
    "category": "Ma catégorie",
  },
  "pages": [
    {
      "title": "Titre de la sous-page",
      "description": "Une courte description de la sous-page"
    }
  ]
}

List translations in the toc.json file

To add links between translations in the footer of each page, the framework needs to know which translations are available. You should add the list in a translations key in the toc.json file, for instance:

{
  "title": "Title of the Roadmap",
  "discourse": {...},
  "pages": [...],
  "translations": [
    {
      "title": "Français",
      "lang": "fr"
    },
    {
      "title": "English",
      "lang": "en"
    },
    {
      "title": "中文",
      "lang": "zh"
    }
  ]
}

The list of translations only needs to appear in the toc.json file, and does not need to appear in the localized toc.xx.json files.