Validate `template.json` files against JSON Schema

[khalidabuhakmeh]

All template.json files in a template should be validated and pass validation against http://json.schemastore.org/template. Currently, there are some validation issues across templates, ranging from the incorrect casing of properties, comments, and more.

Demo Code - Validate Against JSON Schema

This sample uses the following NuGet packages:

• Newtonsoft.Json
• Newtonsoft.Json.Schema
• Spectre.Console (for output)

Note, you'll need to extract all the nupkg templates into a parent directory.

using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http;
using Newtonsoft.Json.Linq;
using Newtonsoft.Json.Schema;
using Spectre.Console;

var templates = Directory.EnumerateFiles(
    "<folder to extracted template packages>", 
    "template.json", 
    SearchOption.AllDirectories);

using var http = new HttpClient();
var jsonSchema = await http.GetStringAsync("https://json.schemastore.org/template");
var schema = JSchema.Parse(jsonSchema);

List<Problem> problems = new();
foreach (var template in templates)
{
    var data = await File.ReadAllTextAsync(template);
    var json = JToken.Parse(data);
    if (!json.IsValid(schema, out IList<string> errors))
    {
        dynamic j = json;
        // just in case name is missing
        var name = (j.name ?? template).ToString();
        problems.Add(new Problem(name, errors));
    }
}

var table = new Table()
    .AddColumn("Name")
    .AddColumn("🚨 Errors");

foreach (var problem in problems)
{
    table.AddRow(
        problem.Name, 
        problem.ErrorMessages.EscapeMarkup()
    );
}

AnsiConsole.Render(table);

public record Problem(string Name, IList<string> Errors)
{
    public string ErrorMessages =>
        Errors.Select(e => $"⁃ {e}\n")
            .Aggregate("", (a, i) => a + i);
}

Thoughts

Currently, I extract the archives (i.e. the nupkg files into a parent directory). Since these are essentially zip files, we could use the ZipFile class in .NET to just pull the template.json files out into memory then parse them, helping avoid an additional step. This would also make the tool more flexible as you could give it a directory or a specific nupkg file and it would work just the same.

License

Note that Newtonsoft.Json.Schema is AGPL or requires a license to be purchased if this tool is commercially available.

[sayedihashimi]

@khalidabuhakmeh I don't think I can use Newtonsoft.Json.Schema, I would have to purchase a license for that. I looked at this when I started working on this tool. I also looked for alternatives and I unfortunately wasn't able to find any that I could use. The license does have an OSS specific support, but if I'm not mistaken I would have to use the AGPL license here to leverage that.

Without using Newtonsoft.Json.Schema, I was able able to run some validation, but the errors that came out were horrible. For example, you would have one issue in your json file, and it would spit out like 10 issues.

[khalidabuhakmeh]

Here is an updated version using NJsonSchema (MIT License). Still using Newtonsoft.Json for parsing the Json but that could also use System.Text.Json.

using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http;
using Newtonsoft.Json.Linq;
using NJsonSchema;
using Spectre.Console;

var templates = Directory.EnumerateFiles(
    "/Users/khalidabuhakmeh/Desktop/untitled folder",
    "template.json", 
    SearchOption.AllDirectories);

using var http = new HttpClient();
var jsonSchema = await http.GetStringAsync("https://json.schemastore.org/template");
var schema = await JsonSchema.FromJsonAsync(jsonSchema);

List<Problem> problems = new();
foreach (var template in templates)
{
    var data = await File.ReadAllTextAsync(template);
    var json = JObject.Parse(data);

    // just in case name is missing
    var name =  json["name"]?.ToString() ?? template;
    var problem = new Problem(name, new List<string>());

    var results = schema.Validate(data);
    if (results.Any())
    {
        var errors = results
            .Select(e => $"{e.Kind} {e.Path} at L#{e.LineNumber} position {e.LinePosition}")
            .ToList();

        problem.Errors.AddRange(errors);
    }

    if (problem.Errors.Any())
    {
        problems.Add(problem);
    }
}

var table = new Table()
    .AddColumn("Name")
    .AddColumn("🚨 Errors");

foreach (var problem in problems)
{
    table.AddRow(
        problem.Name, 
        problem.ErrorMessages.EscapeMarkup()
    );
}

AnsiConsole.Render(table);

public record Problem(string Name, List<string> Errors)
{
    public string ErrorMessages =>
        Errors.Select(e => $"⁃ {e}\n")
            .Aggregate("", (a, i) => a + i);
}

With the output of.

┌────────────────────────┬──────────────────────────────────────────────────────────┐
│ Name                   │ 🚨 Errors                                                │
├────────────────────────┼──────────────────────────────────────────────────────────┤
│ Razor Component        │ ⁃ PropertyRequired #/shortName at L#1 position 1         │
│                        │                                                          │
│ Protocol Buffer File   │ ⁃ PropertyRequired #/shortName at L#1 position 1         │
│                        │                                                          │
│ Blazor WebAssembly App │ ⁃ ArrayItemNotValid #/postActions[1] at L#504 position 5 │
│                        │                                                          │
│ Blazor WebAssembly App │ ⁃ ArrayItemNotValid #/postActions[1] at L#504 position 5 │
│                        │                                                          │
│ Solution File          │ ⁃ PropertyRequired #/tags at L#1 position 1              │
│                        │                                                          │
│ Razor Component        │ ⁃ PropertyRequired #/shortName at L#1 position 1         │
│                        │                                                          │
│ Protocol Buffer File   │ ⁃ PropertyRequired #/shortName at L#1 position 1         │
│                        │                                                          │
│ Solution File          │ ⁃ PropertyRequired #/tags at L#1 position 1              │
│                        │                                                          │
│ Solution File          │ ⁃ PropertyRequired #/tags at L#1 position 1              │
│                        │                                                          │
│ Blazor WebAssembly App │ ⁃ ArrayItemNotValid #/postActions[1] at L#472 position 5 │
│                        │                                                          │
│ Razor Component        │ ⁃ PropertyRequired #/shortName at L#1 position 1         │
│                        │                                                          │
│ Protocol Buffer File   │ ⁃ PropertyRequired #/shortName at L#1 position 1         │
│                        │                                                          │
└────────────────────────┴──────────────────────────────────────────────────────────┘

The output has duplicates because I exported multiple target frameworks out into directories. An updated version might want to also have a column with the framework version.