Support multiple documents in one file

[simt2]

Displays an error yaml: expected a single document in the stream, but found more when editing a file containing multiple documents:

---
foo: bar
---
hue: hue

[djabraham]

Thanks for the heads up. I've never heard of multiple documents. Looks like a recent addition to the spec, perhaps?

http://yaml.org/spec/1.1/

vs.

http://yaml.org/spec/1.2/spec.html

Might not be all that difficult to support multiple docs.

[tamas-molnar]

I use this extension to validate Kubernetes deployment files, which contain multiple docs, so support for this would be really useful! 👍

[simt2]

This is exactly what I'm using it for! Unfortunately I'm not familiar enough with JS/TS to try to implement this myself.

[djabraham]

Where did you guys find a spec for those? I looked a while ago, and they were scarce or incomplete.

[simt2]

There is no actual "Kubernetes deployment file", just deployments, services and other objects defined by the Kubernetes API. But it is common to combine objects belonging to the same application into one multi-document YAML file.

[apkd]

Confirming this issue with documents containing YAML Streams. From the 1.0 spec:

A YAML character stream may contain several documents. Each document is completely independent from the rest.

Basically, you stick several YAML documents separated by "---". Works with the YAML parser I'm using (sharpyaml) and I'd love to have automatic schema validation in vscode (the documents are human-written).

More information: http://yaml.org/spec/1.2/spec.html#YAML This feature is present all the way back in spec 1.0: http://yaml.org/spec/1.0/#id2561559

[djabraham]

Oh, sorry. A comment I made got misconstrued earlier. When I asked about a spec, I should have said schema. I use a JSON schema to validate open-api/swagger docs. When someone mentioned Kubernetes, I thought maybe you also had a schema.

The reason I made this plugin, was to get feedback on doc whenever I had an unacceptable tag. In other words, the schema validates semantics and relationships for specific type of document. Whereas, the built-in colorizer only checks the generic structure applicable to all types.

I could not find a schema for Kubernetes. I started playing with creating one automatically, by feeding them into a reverse engineer tool. I cannot remember what the tool was called, but I located this example schema for Kubernetes Job manifests in a local project. The schema would have to be referenced in your .vscode/settings.json like so:

settings.json

// Place your settings in this file to overwrite the default settings
{
  "json.schemas": [
    {
      "fileMatch": [
        "**/default.json"
      ],
      "url": "./src/specs/schema/sources/JsonSchema.d4.json"
    },
    {
      "fileMatch": [
        "**/swagger.yaml",
        "**/swagger.json"
      ],
      "url": "http://json.schemastore.org/swagger-2.0"
    },
    {
      "fileMatch": [
        "**/*job.yaml"
      ],
      "url": ".vscode/schema-kube-job.json"
    }
  ]
}

schema-kube-job.json

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "apiVersion": {
      "type": "string"
    },
    "kind": {
      "type": "string"
    },
    "metadata": {
      "type": "object",
      "properties": {
        "annotations": {
          "type": "object",
          "properties": {
            "contact": {
              "type": "string"
            },
            "description": {
              "type": "string"
            }
          },
          "required": [
            "contact",
            "description"
          ]
        },
        "name": {
          "type": "string"
        }
      },
      "required": [
        "annotations",
        "name"
      ]
    },
    "spec": {
      "type": "object",
      "properties": {
        "activeDeadlineSeconds": {
          "type": "integer"
        },
        "template": {
          "type": "object",
          "properties": {
            "metadata": {
              "type": "object",
              "properties": {
                "labels": {
                  "type": "object",
                  "properties": {
                    "app": {
                      "type": "string"
                    },
                    "component": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "app",
                    "component"
                  ]
                }
              },
              "required": [
                "labels"
              ]
            },
            "spec": {
              "type": "object",
              "properties": {
                "restartPolicy": {
                  "type": "string"
                },
                "volumes": {
                  "type": "array",
                  "items": {
                    "type": "object",
                    "properties": {
                      "name": {
                        "type": "string"
                      },
                      "hostPath": {
                        "type": "object",
                        "properties": {
                          "path": {
                            "type": "string"
                          }
                        },
                        "required": [
                          "path"
                        ]
                      }
                    },
                    "required": [
                      "name",
                      "hostPath"
                    ]
                  }
                },
                "containers": {
                  "type": "array",
                  "items": {
                    "type": "object",
                    "properties": {
                      "name": {
                        "type": "string"
                      },
                      "image": {
                        "type": "string"
                      },
                      "command": {
                        "type": "array",
                        "items": {
                          "type": "string"
                        }
                      },
                      "volumeMounts": {
                        "type": "array",
                        "items": {
                          "type": "object",
                          "properties": {
                            "name": {
                              "type": "string"
                            },
                            "mountPath": {
                              "type": "string"
                            }
                          },
                          "required": [
                            "name",
                            "mountPath"
                          ]
                        }
                      },
                      "env": {
                        "type": "array",
                        "items": {
                          "type": "object",
                          "properties": {
                            "name": {
                              "type": "string"
                            },
                            "value": {
                              "type": "string"
                            }
                          },
                          "required": [
                            "name",
                            "value"
                          ]
                        }
                      }
                    },
                    "required": [
                      "name",
                      "image",
                      "command",
                      "volumeMounts",
                      "env"
                    ]
                  }
                }
              },
              "required": [
                "restartPolicy",
                "volumes",
                "containers"
              ]
            }
          },
          "required": [
            "metadata",
            "spec"
          ]
        }
      },
      "required": [
        "activeDeadlineSeconds",
        "template"
      ]
    }
  },
  "required": [
    "apiVersion",
    "kind",
    "metadata",
    "spec"
  ]
}

[apkd]

I do not use this extension for Kubernetes but nevertheless, if people are putting several documents in one file using YAML streams then unfortunately the extension will freak out.

Regrettably, I haven't been able to locate any other decent editor with JSON schema validation for YAML. VSCode + this extension + YAML streams support = ❤️

[johlrich]

Any updates on this one? Is this just a matter of removing the alerting to this rule or would there be other considerations needed so the documents are truly treated separately?

[djabraham]

I have to check several things. I have some time this weekend (finally)

[djabraham]

I started work on this. I had to update to a newer extension framework first. Should have something soon.

[adamvoss]

I am not sure I understand how this would be expected to work. What is the expected behavior of multiple documents in a single file? Would they be assumed to all adhere to the same schema? Or is it more that the schema validation gets in the way with multiple documents and should just be disabled when present?

[djabraham]

Filename based schema selection actually seems like a bit of an anti-pattern to me. If you were authoring a schema in JSON, then you would start out with a reference to the schema that is used to validate your schema, like so:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "properties": {
    "apiVersion": {
            "type": "string"
        },
        "kind": {
            "type": "string"
        }...

When you follow that trail, it's interesting to note that this schema actually references itself. Because "of course".

{
    "id": "http://json-schema.org/draft-04/schema#",
    "$schema": "http://json-schema.org/draft-04/schema#",
    "description": "Core schema meta-schema",
    "definitions": {
        "schemaArray": {
            "type": "array",
            "minItems": 1,
            "items": { "$ref": "#" }
        }...

I believe this $schema keyword is reserved for schema itself, so others have suggested work-arounds or extensions to authoring self-describing json, as shown here. But until something official arises, I would assume by default that all documents in a given file adhear to the same schema. If not, how would a consumer parse them?

Kubernetes deployment templates are declarative YAML constructs consumed by the kubectl utility. While they have structural similarities in many cases, they are formatted according to the requirements of a specific CLI command invoked. In this case, I believe a particular command expects a singular file (or chained file) structure.

[adamvoss]

I took a stab at this. If I suppress some errors given by the parser library, validation and hover appear to work:

Without the errors suppressed, everything after the first document shows with red underlines. I wouldn't release this with it suppressing errors, because in other contexts the errors are legitimate. Once mulesoft-labs/yaml-ast-parser#20 and mulesoft-labs/yaml-ast-parser#21 are fixed and has a new release out on npm, it appears this feature should be doable.

[adamvoss]

Good news: a fix (mulesoft-labs/yaml-ast-parser#24) has been accepted. As I understand it, down-stream changes are needed to some RAML libraries before it will have a new, published build, but once one is published it should not take much to get an updated VS Code Extension available.

[adamvoss]

This is available with vscode-yaml as of version 0.0.7. You may switch to that extension to get this functionality.

[0x80]

@adamvoss The extension is not published it seems. Any news on this?

[djabraham]

Closing, this extension was retired.