search

dotnet / docfx

Static site generator for .NET API documentation.
https://dotnet.github.io/docfx/
MIT License
4.06k stars 864 forks source link

Support schema driven documents from JSON schema #4067

Closed yufeih closed 5 years ago

yufeih commented 5 years ago

Phases

Feature Scope

Json schema validation

Json schema transform

Template integration

Xref handling

Landing data handling

Future clean up

OsmondJiang commented 5 years ago

Xref Handling

Schema Example:

{  
   "properties":{  
      "description":{  
         "type":"string"
      }
      ...
   },
   "xrefProperties":[  
      "description",
      "remarks",
      "name",
      "type"
   ]
}

Xref transform will build all properties(transformed) defined in xrefProperties into xref spec, constitute a xref map group by uid:

{  
   "uid-1":{  
      "href":"uid-href",
      "monikers":[  
         "dotnet-2.0"
      ],
      "description":"uid description"
   }
}

Schema Example:

{  
   "properties":{  
      "name":{  
         "type":"string",
         "contentType":"xref"
      }
   }
}

Schema Transform Input

{  
   "name":"System.String"
}

Schema Transform Output

{  
   "name":{  
      "uid":"System.String",
      "href":"xxxxx",
       "displayName":"System String"
   }
}

Template(before)

<xref uid="{{ uid }}" template="partials/dotnet/xref-name-summary.tmpl" />

Template(after)

{{> partials/dotnet/xref-name-summary.tmpl }}
mingwli commented 5 years ago

Legacy Template Integration

Scope

Currently the template exists only for legacy build, the ideal template system for v3 needs further design and isn't in the legacy integration stage scope.

Existing types of template

After an OutputModel(PageModel) is generated as input, three kinds of templates may be applied and there output is listed below:

applicable template direct output v3 json(true) output v3 json(false) output v2 output
{schema}.html.primary.{js,tmpl} String *.json *.html *.raw.page.json(with mta.json.js' output)
*.mta.json.js JObject ❌ ❌ *.mta.json
*.json.js JObject ❌ ❌ *.json

For schema with html template, the input document is treated as a PageSchema(isPage == true), otherwise it's treated as DataSchema. The mta template is optional for both schema type.

PageSchema and DataData output

Output files

Schema document.raw.page.json document.json document.mta.json
Page ✔ ❌ ⭕
Data ❌ ✔ ⭕

For both Page & Data schema, if a mta template exists, an document.mta.json will be generated. Note that although mta template is optional, existing PageSchema all have a mta template.

PageSchema (Conceptual + SDP page)

PageModel -> Apply mta template -> rawMetadata PageModel -> Apply html template(js transform + mustache) -> content html

TemplateModel:
  content: # html
  outputRootRelativePath:
  pageMetadata: # html
  rawMetadata: # generated by `mta` template transform
  themesRelativePathToOutputRoot:

* current v3 code doesn't appy conceptual html template, but does the logic in html post process

e.g. APIConnector

intput:

https://github.com/MicrosoftDocs/DevSandbox/blob/master/devsandbox/APIConnector/youtube.yml ```yaml ### YamlMime:YamlDocument documentType: APIConnector actions: [] triggers: - parameters: - required: true summary: Channel Id type: string description: Pick a channel. summary: When a video is uploaded by a channel description: This operation triggers when a new video is uploaded by a channel. ... definitions: - properties: - summary: Videos type: array of Video description: List of all videos. path: items summary: VideoList description: A list of YouTube videos. ... title: YouTube description: 'YouTube allows billions of people to discover, watch and share videos.' status: Preview version: '1.2' ... ```

output:

*.raw.page.json ```json { "content": "...", "outputRootRelativePath": "../", "pageMetadata": "...", "rawMetadata": { "metadata": null, "title": "YouTube (Preview) | Microsoft Docs", "description": "YouTube allows billions of people to discover, watch and share videos.", "breadcrumb_path": "/devsandbox/breadcrumb.json", "apiPlatform": "dotnet", "pdf_url_template": "https://docs.microsoft.com/pdfstore/en-us/VS.core-docs/{branchName}{pdfName}", "open_to_public_contributors": true, "search.ms_sitename": "Docs", "search.ms_docsetname": "devsandbox", "_op_canonicalUrlPrefix": "https://docs.microsoft.com/en-us/devsandbox/", ... }, "themesRelativePathToOutputRoot": "_themes/" } ``` *.mta.json ```json { "metadata": null, "title": "YouTube (Preview) | Microsoft Docs", "description": "YouTube allows billions of people to discover, watch and share videos.", "breadcrumb_path": "/devsandbox/breadcrumb.json", "apiPlatform": "dotnet", "pdf_url_template": "https://docs.microsoft.com/pdfstore/en-us/VS.core-docs/{branchName}{pdfName}", "open_to_public_contributors": true, "search.ms_sitename": "Docs", "search.ms_docsetname": "devsandbox", "locale": "en-us", "site_name": "Docs", "search.ms_product": "MSDN", "depot_name": "MSDN.devsandbox", "author": "adkinn", "updated_at": "2019-05-29 05:27 PM", "gitcommit": "https://github.com/MicrosoftDocs/DevSandbox/blob/cdf1f17d12423de8645537b63980a9bcad7bd239/devsandbox/APIConnector/youtube.yml", "original_content_git_url": "https://github.com/MicrosoftDocs/DevSandbox/blob/master/devsandbox/APIConnector/youtube.yml", "content_git_url": "https://github.com/MicrosoftDocs/DevSandbox/blob/master/devsandbox/APIConnector/youtube.yml", "document_id": "3b627de1-9fdc-714b-d4b2-ed05da662b30", "document_version_independent_id": "067f5dbc-5287-2f6d-12af-a46b6fac44bf", "layout": "Reference", "page_type": "apiconnector", "canonical_url": "https://docs.microsoft.com/en-us/devsandbox/apiconnector/youtube", "toc_asset_id": "toc.json", "toc_rel": "../toc.json", "is_dynamic_rendering": true } ```

DataSchema (SDP data)

PageModel -> Apply mta template

* Only one DataSchema(achievements) has mta template, and its only used for generating an literal object {"page_Type":"learn", "page_kind":"achievements"}. Since other learn's DataSchema doesn't have it, it may possibly be of no usage. * Learn repo isn't in current phase, hence doesn't need to be taken into consideration. DataSchema DO NOT need to apply mta template.

e.g. ContextObject input: 

### YamlMime:ContextObject
brand: azure
breadcrumb_path: ../breadcrumb/TOC.yml
toc_rel: ../TOC.yml
output: *.json

```json { "metadata": { "pdf_name": "topics/high-performance-computing/context/hpc-context.pdf", "pdf_absolute_path": "/e2e-azure-architecture-center/opbuildpdf/topics/high-performance-computing/context/hpc-context.pdf" }, "brand": "azure", "breadcrumb_path": "../breadcrumb/TOC.json", "toc_rel": "../TOC.json", "feedback_github_repo": "mspnp/architecture-center", ... "search.ms_product": "MSDN", "depot_name": "MSDN.e2e-azure-architecture-center", "author": "adamboeglin", "_op_gitContributorInformation": { "author": { "profile_url": "https://github.com/adamboeglin", "display_name": "Adam Boeglin", "name": "adamboeglin", "id": "40869465" }, ... }, "document_id": "efc37152-c511-e306-b1f4-770f70bf9932", "document_version_independent_id": "0f8de835-d8da-4f38-2dfd-65de0559add1" } ```

Other Info

DataFlow

  | ✔ Read SDP file schema |
            🔻
            🔻
  | ⭕ Resolve json schema from template |
            🔻
            🔻
  | ✔ Schema validate & transform => raw model |
            🔻
            🔻
  | ⭕ Resolve template & Run Jint + Mustache |
  | ✔ I schema.json.js => '.json'|
  | ❌ I schema.json.tmpl |
  | ✔ II schema.mta.json.js => '.mta.json' |
  | ❌ II schema.mta.json.tmpl |
  | ✔ III schema.html.primary.js + schema.html.primary.tmpl => 'html content' | ➡ OutputModel.content ➡ 'raw.page.json'
  (* 'schema.json.js' is only used with 'schema.mta.json.js')

Existing *.json.js logic

acheievements: keep(exams, countries, metadata)
EmailOptInPreferences: keep(preferences, metadata)
ExamPricing: keep(exams, contries, metadata)
ModuleAvailability: keep(disableAllLabs, disableAllSandboxes, disabledLabs, disabledSandboxes)
SurveyQuestions: keep(questions, metadata)
ZonePivotGroups: keep(groups, metadata)

ContextObject: resetKeys(product_xxx, version, locale)
Quiz: resetKeys()

FormModel: falsy output => change to array
herohua commented 5 years ago

@OsmondJiang @mingwli We may need a separate item on bookmark validation for SDP.

herohua commented 5 years ago

phase 1 is now being tracked by https://dev.azure.com/ceapex/Engineering/_workitems/edit/95863

phase 2: https://dev.azure.com/ceapex/Engineering/_workitems/edit/102269