Assemblies health check

[abjerner]

I haven't really worked on this since the retreat, but the idea was that we could have a health check that would validate assemblies against a list (one or more feeds) of known vulnerabilities (or just regular warnings and errors).

Primarily this would be the DLLs that ship with Umbraco (as discussed with @nul800sebastiaan, the Umbraco feed should most likely live on Our Umbraco - hence an issue on this repository), but developers might be able to create their own additional feed. So the first step would be to settle on a format for the feed - and as everything ise JSON these that, that format might as well be described by a JSON scheme:

{
  "title": "A schema for vulnerabilities and issues.",
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "definitions": {
    "author": {
      "type": "object",
      "description": "This describes details about the author.",
      "required": [ "name" ],
      "additionalProperties": false,
      "properties": {
        "name": {
          "type": "string",
          "description": "The name of the author."
        },
        "url": {
          "type": "string",
          "description": "The URL of the author's website."
        }
      }
    },
    "product": {
      "type": "object",
      "required": [ "name", "issues" ],
      "description": "This describes details about a product.",
      "additionalProperties": false,
      "properties": {
        "name": {
          "type": "string",
          "description": "The name of the product."
        },
        "url": {
          "type": "string",
          "description": "The URL of the website of the product."
        },
        "issues": {
          "type": "array",
          "description": "An array of known issues for the product.",
          "uniqueItems": true,
          "items": {
            "$ref": "#/definitions/issue"
          }
        }
      }
    },
    "issue": {
      "type": "object",
      "required": [ "type", "severity", "name", "assemblies" ],
      "description": "This describes details about an issue of a product.",
      "additionalProperties": false,
      "properties": {
        "type": {
          "type": "string",
          "enum": [ "security", "major", "minor", "patch" ]
        },
        "severity": {
          "type": "string",
          "enum": [ "high", "medium", "low" ]
        },
        "date": {
          "type": "string",
          "format": "date",
          "description": "The date of the issue."
        },
        "name": {
          "type": "string",
          "description": "The name of the issue."
        },
        "url": {
          "type": "string",
          "description": "The URL with information about the issue."
        },
        "assemblies": {
          "type": "array",
          "description": "...",
          "items": {
            "$ref": "#/definitions/assembly"
          }
        }
      }
    },
    "assembly": {
      "type": "object",
      "required": [ "alias", "version" ],
      "additionalProperties": false,
      "properties": {
        "alias": {
          "type": "string",
          "description": "The alias/name of the assembly."
        },
        "version": {
          "type": "string",
          "description": "The version range affected by the issue."
        }
      }
    }
  },
  "properties": {
    "author": {
      "type": "object",
      "description": "The author of the feed.",
      "$ref": "#/definitions/author"
    },
    "products": {
      "type": "array",
      "description": "A list of product.",
      "uniqueItems": true,
      "items": {
        "$ref": "#/definitions/product"
      }
    }
  }
}

Given this JSON scheme, the Umbraco feed could then look as:

{
  "author": {
    "name": "Umbraco",
    "url": "https://umbraco.com/"
  },
  "products": [
    {
      "name": "Umbraco Forms",
      "url": "https://our.umbraco.org/projects/developer-tools/umbraco-forms/",
      "issues": [
        {
          "type": "security",
          "severity": "high",
          "date": "2016-01-27",
          "name": "Umbraco Forms Security Notice",
          "url": "https://umbraco.com/blog/umbraco-forms-security-notice/",
          "assemblies": [
            {
              "type": "assembly",
              "alias": "Umbraco.Forms.Core",
              "version": "[4.0,4.4.2)"
            }
          ]
        },
        {
          "type": "security",
          "severity": "high",
          "date": "2017-02-28",
          "name": "Security advisory: Update Umbraco Forms immediately",
          "url": "https://umbraco.com/blog/security-advisory-update-umbraco-forms-immediately/",
          "assemblies": [
            {
              "type": "assembly",
              "alias": "Umbraco.Forms.Core",
              "version": "[4.0,4.4.2)"
            }
          ]
        },
        {
          "type": "security",
          "severity": "high",
          "date": "2018-05-15",
          "name": "Umbraco Forms Security update",
          "url": "https://umbraco.com/blog/umbraco-forms-security-update/",
          "assemblies": [
            {
              "type": "assembly",
              "alias": "Umbraco.Forms.Core",
              "version": "[4.4.2,4.4.7)"
            },
            {
              "type": "assembly",
              "alias": "Umbraco.Forms.Core",
              "version": "[6.0.0,6.0.8)"
            },
            {
              "type": "assembly",
              "alias": "Umbraco.Forms.Core",
              "version": "[7.0.0,7.0.3)"
            }
          ]
        }
      ]
    }
  ]
}

The feed could also contain reports of more than just vulnerabilities. For instance when Instagram made a breaking change in their API, it affected my Skybrud.Social package. So if an affected version of this in installed, it could trigger a warning. So at Skybrud we could have a feed for our packages:

{
  "author": {
    "name": "Skybrud.dk",
    "url": "https://www.skybrud.dk/"
  },
  "products": [
    {
      "name": "Skybrud.Social",
      "url": "https://social.skybrud.dk/",
      "issues": [
        {
          "type": "warning",
          "severity": "medium",
          "date": "2017-04-27",
          "name": "Skybrud.Social + Instagram carousels",
          "description": "Due to a change in the Instagram API an update Skybrud.Social is required to avoid errors when showing content from Instagram.",
          "assemblies": [
            {
              "type": "assembly",
              "alias": "Skybrud.Social",
              "version": "0.9.6)"
            }
          ]
        }
      ]
    }
  ]
}

Feed

At the root of the feed are only two properties - author and products:

• author
  Describes the author of the feed (eg. Umbraco for the Umbraco feed).
• products
  An array of products (see below).

Product

• name
  Describes the name of the product - eg. Umbraco CMS or Umbraco Forms.
• url
  URL to a website or page about the product.
• issues
  An array of known issues of the particular product.

Issue

• type
  Describes the type of the issue. This could be a enum with the following values:
  • security
  • warning
  • patch?
  • info?
• severity
  Describes the severity/impact of an issue. In a similar way to type, severity could be a enum with the following values:
  • high
  • normal
  • medium
  • low
• date
  Ther date the issue was created/reported.
• name
  The name/title of the issue.
• url
  The URL to a page with information about the issue.
• assemblies
  Array of affected assemblies (see below).

Assembly

• type
  Probably always assembly - can't remember why I included this one.
• alias
  The alias (or filename without .dll) of the assembly.
• version
  The affected version or version range. Follows the format of the NuGet specification.

With the schema proposal above, we're only looking at assemblies, but I suppose there could be issues with other files as well. This was the case in some of the earlier versions of Umbraco - eg. where the fix was to either delete or update the affected files (meaning we'd have no change in Umbraco version to check against). So perhaps there should also be an option to list affected files (which may again be based on either file checksums or whether certain values are part of the file).

An assembly also has both a version and a file version, so perhaps it should be possible check against either value. I think some of the Microsoft DLLs have been a bit bad with their version numbers in the bad, although I can't remember the exact cases.

[abjerner]

@nul800sebastiaan as we talked about at the meetup, for handling the last two security issues where there are no change in version numbers if you apply the hotfix, it might be good with a way to handle this in the format as well.

As we can't really check whether the hotfix has been applied, the best I can think of is to introduce a hotfix property for the issues, which then will indicate whether a hotfix is available for the issue. If this is the case, the issue will show up as Indeterminate (normal would be Affected) when Umbraco is within the affected version range.

It would also make sense to somehow be able to dismiss an issue where a hotfix is available (eg. when you have applied the hotfix) so the issue will no longer show up. But this last part more something to be handled in the code, and not in the JSON schema.

Does this make sense?

[umbrabot]

Hiya @abjerner,

Just wanted to let you know that we noticed that this issue got a bit stale and might not be relevant any more.

We will close this issue for now but we're happy to open it up again if you think it's still relevant (for example: it's a feature request that's not yet implemented, or it's a bug that's not yet been fixed).

To open it this issue up again, you can write @umbrabot still relevant in a new comment as the first line. It would be super helpful for us if on the next line you could let us know why you think it's still relevant.

For example:

@umbrabot still relevant This bug can still be reproduced in version 8.9.0

This will reopen the issue in the next few hours.

Thanks, from your friendly Umbraco GitHub bot :robot: :slightly_smiling_face:

[abjerner]

@umbrabot still relevant

[umbrabot]

Hiya @abjerner,

Just wanted to let you know that we noticed that this issue got a bit stale and might not be relevant any more.

We will close this issue for now but we're happy to open it up again if you think it's still relevant (for example: it's a feature request that's not yet implemented, or it's a bug that's not yet been fixed).

To open it this issue up again, you can write @umbrabot still relevant in a new comment as the first line. It would be super helpful for us if on the next line you could let us know why you think it's still relevant.

For example:

@umbrabot still relevant This bug can still be reproduced in version x.y.z

This will reopen the issue in the next few hours.

Thanks, from your friendly Umbraco GitHub bot :robot: :slightly_smiling_face: