search

Azure / azure-resource-manager-schemas

Schemas used to author and validate Resource Manager Templates. These schemas power the intellisense and syntax completion in our ARM Tools VSCode extension, as well as the Export Template API
MIT License
609 stars 516 forks source link

Inconsistencies in 2015-01-01 deploymentTemplate #1048

Open matthchr opened 4 years ago

matthchr commented 4 years ago

There are a number of issues in the 2015-01-01 template which I've happened to run across after running some automated tooling across it.

Resources without Name

There are 128 different resources referenced which do not have a Name - as far as I can tell see here, Name is a required property for the ARM JSON schema. Without a name, it's not actually possible to submit a resource to ARM, you get an error:

The request content was invalid and could not be deserialized: 'Required property 'name' not found in JSON. Path 'properties.template.resources[0]', line 19, position 9.'.

Here are some examples (not all 128 of them):

  1. https://schema.management.azure.com/schemas/2016-06-01/Microsoft.RecoveryServices.json#/resourceDefinitions/vaults
  2. https://schema.management.azure.com/schemas/2016-06-01/Microsoft.RecoveryServices.json#/resourceDefinitions/vaultsExtendedInformation
  3. https://schema.management.azure.com/schemas/2015-06-01/Microsoft.KeyVault.json#/resourceDefinitions/secrets
  4. https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites

I specifically tested deploying a 2015-06-01 secret into a KeyVault and confirmed that even though the schema is missing name, it is indeed required and the template deployment doesn't succeed without it.

There are a bunch of others spread across the following specs: microsoft.recoveryservices/v20160601 microsoft.keyvault/v20150601 microsoft.keyvault/v20161001 microsoft.web/v20140601 microsoft.web/v20150801 microsoft.domainregistration/v20150401 microsoft.certificateregistration/v20150801 microsoft.web/v20160301 microsoft.web/v20160801 microsoft.web/v20160901 microsoft.insights/v20140401 microsoft.visualstudio/v20140226 microsoft.notificationhubs/v20150401 microsoft.network/v20151101 microsoft.network/v20170301 microsoft.network/v20170501 microsoft.storage/v20150801 microsoft.storage/v20160101 microsoft.storage/v20180201 microsoft.storage/v20180701 microsoft.storage/v20181101 microsoft.storage/v20190401 microsoft.storage/v20190601 microsoft.compute/v20150801 microsoft.network/v20160401 microsoft.logic/v20160601 microsoft.logic/v20161001 microsoft.logic/v20170701 microsoft.web/v20160601 microsoft.media/v20151001 microsoft.servicefabric/v20160301 microsoft.authorization/v20160901 microsoft.apimanagement/v20170301 microsoft.apimanagement/v20180101 microsoft.apimanagement/v20190101 microsoft.compute/v20170330 microsoft.insights/v20170401 microsoft.timeseriesinsights/v20171115 microsoft.compute/v20171201 microsoft.insights/v20171001 microsoft.recoveryservices/v20161201 microsoft.security/v20180601 microsoft.insights/v20190301 microsoft.web/v20180201 microsoft.web/v20181101 microsoft.web/v20181101

Resources which are defined in referenced schemas but aren't actually referenced directly in the root 2015-01-01 template spec

There are 113 resources which aren't referenced in the 2015-01-01 deployment template but probably should be.

Here are some examples:

  1. https://schema.management.azure.com/schemas/2019-02-01/Microsoft.Network.json#/resourceDefinitions/loadBalancers_inboundNatRules
  2. https://schema.management.azure.com/schemas/2018-01-01/Microsoft.Network.json#/resourceDefinitions/expressRouteCircuits_peerings
  3. https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/networkSecurityGroups_securityRules

There are others that aren't in Microsoft.Network as well.

Since the readme specifically says:

Ensure that any new resource types that you are adding have been added to the following top-level template schemas schemas/2014-04-01-preview/deploymentTemplate.json schemas/2015-01-01/deploymentTemplate.json schemas/2019-04-01/deploymentTemplate.json

This feels like an oversight.

matthchr commented 4 years ago

For the second category of problem (resources which are defined but not referenced directly), provided there is agreement that the issue is just that they were missed it should be easy enough to fix (just make an update to the root schema to include the missed resource).

For the first problem it's a bit more complicated as I am not sure how the referenced schemas are generated and why name was missed on a bunch of them. Can you confirm that name is 100% required for every resource inside resourceDefinitions? (it seems like it is).

jorgecotillo commented 4 years ago

@neilpeterson if a schema, say KeyVault, does not specify "name" property as required, and I create an ARM template and validate it with the ARM Tools VSCode extension, is the extension going to complain because of a missing required property? If yes, is it looking at a different definition inside the schema? perhaps a shared definition?

neilpeterson commented 4 years ago

@jorgecotillo I am not positive. In a quick test with Microsoft.KeyVault/vaults@2015-06-01 I see that a warning is thrown for the missing name however the location is incorrect which is atypical for schema validation.

@alexgav do you have any thoughts on this?

image

matthchr commented 4 years ago

For the second item, just to illustrate the issue I looked at: https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/networkSecurityGroups_securityRules

This resource +api version is not in the top level template, but other api versions are, for example: https://schema.management.azure.com/schemas/2015-06-15/Microsoft.Network.json#/resourceDefinitions/networkSecurityGroups_securityRules

The 2017-06-01 version is definitely valid though, as I deployed the following ARM template and it works:

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {},
  "variables": {},
  "resources": [
    {
      "type": "Microsoft.Network/networkSecurityGroups/securityRules",
      "apiVersion": "2017-06-01",
      "name": "matthchr-testnsg/securityrule1",
      "properties": {
          "description": "A quick description",
          "protocol": "Tcp",
          "sourcePortRange": "*",
          "destinationPortRange": "*",
          "destinationAddressPrefix": "VirtualNetwork",
          "sourceAddressPrefix": "Internet",
          "access": "Deny",
          "priority": 100,
          "direction": "Inbound"
      }
    }
  ]
}

I believe the same situation is happening across many other resources+version pairs too - certain ones are left out (but actually work in ARM). As I mentioned I think there are ~100 such combinations at least that should be in the spec but aren't.

There's even a section in the schema where other networking resources from the 2017-06-01 networking API version are defined, but for some reason networkSecurityGroups_securityRules isn't there with them (even though if you go to the referenced Microsoft.Network.json it's in there).

see: 

                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/publicIPAddresses" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/virtualNetworks" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/loadBalancers" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/networkSecurityGroups" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/networkInterfaces" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/routeTables" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/applicationGateways" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/connections" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/localNetworkGateways" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/virtualNetworkGateways" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/virtualNetworks_subnets" },
                  { "$ref": "https://schema.management.azure.com/schemas/2017-06-01/Microsoft.Network.json#/resourceDefinitions/virtualNetworks_virtualNetworkPeerings" }
matthchr commented 4 years ago

As discussed offline, sharing the resources which I know are impacted by each of these issues.

Note that this is for the 2015-01-01 spec, I don't have the data for the other specs.

Name missing: 

https://schema.management.azure.com/schemas/2016-06-01/Microsoft.RecoveryServices.json#/resourceDefinitions/vaults
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.RecoveryServices.json#/resourceDefinitions/vaults_extendedInformation
https://schema.management.azure.com/schemas/2015-06-01/Microsoft.KeyVault.json#/resourceDefinitions/secrets
https://schema.management.azure.com/schemas/2016-10-01/Microsoft.KeyVault.json#/resourceDefinitions/vaults_accessPolicies
https://schema.management.azure.com/schemas/2014-06-01/Microsoft.Web.json#/resourceDefinitions/certificates
https://schema.management.azure.com/schemas/2014-06-01/Microsoft.Web.json#/resourceDefinitions/serverfarms
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.Web.json#/resourceDefinitions/certificates
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.Web.json#/resourceDefinitions/serverfarms
https://schema.management.azure.com/schemas/2015-04-01/Microsoft.DomainRegistration.json#/resourceDefinitions/domains
https://schema.management.azure.com/schemas/2015-04-01/Microsoft.DomainRegistration.json#/resourceDefinitions/domains_domainOwnershipIdentifiers
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.CertificateRegistration.json#/resourceDefinitions/certificateOrders
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.CertificateRegistration.json#/resourceDefinitions/certificateOrders_certificates
https://schema.management.azure.com/schemas/2016-03-01/Microsoft.Web.json#/resourceDefinitions/certificates
https://schema.management.azure.com/schemas/2016-03-01/Microsoft.Web.json#/resourceDefinitions/csrs
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_config
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_deployments
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_domainOwnershipIdentifiers
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_hostNameBindings
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_hybridconnection
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_hybridConnectionNamespaces_relays
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_instances_deployments
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_premieraddons
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_publicCertificates
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_config
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_deployments
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_domainOwnershipIdentifiers
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_hostNameBindings
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_hybridconnection
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_hybridConnectionNamespaces_relays
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_instances_deployments
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_premieraddons
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_publicCertificates
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_virtualNetworkConnections
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_virtualNetworkConnections_gateways
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_sourcecontrol
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_sourcecontrol
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_virtualNetworkConnections
https://schema.management.azure.com/schemas/2016-08-01/Microsoft.Web.json#/resourceDefinitions/sites_virtualNetworkConnections_gateways
https://schema.management.azure.com/schemas/2016-09-01/Microsoft.Web.json#/resourceDefinitions/hostingEnvironments
https://schema.management.azure.com/schemas/2016-09-01/Microsoft.Web.json#/resourceDefinitions/hostingEnvironments_workerPools
https://schema.management.azure.com/schemas/2016-09-01/Microsoft.Web.json#/resourceDefinitions/hostingEnvironments_multiRolePools
https://schema.management.azure.com/schemas/2016-09-01/Microsoft.Web.json#/resourceDefinitions/serverfarms
https://schema.management.azure.com/schemas/2016-09-01/Microsoft.Web.json#/resourceDefinitions/serverfarms_virtualNetworkConnections_gateways
https://schema.management.azure.com/schemas/2016-09-01/Microsoft.Web.json#/resourceDefinitions/serverfarms_virtualNetworkConnections_routes
https://schema.management.azure.com/schemas/2014-04-01/Microsoft.Insights.json#/resourceDefinitions/alertrules
https://schema.management.azure.com/schemas/2014-04-01/Microsoft.Insights.json#/resourceDefinitions/components
https://schema.management.azure.com/schemas/2014-04-01/Microsoft.Insights.json#/resourceDefinitions/autoscalesettings
https://schema.management.azure.com/schemas/2014-04-01/Microsoft.Insights.json#/resourceDefinitions/webtests
https://schema.management.azure.com/schemas/2014-02-26/microsoft.visualstudio.json#/resourceDefinitions/account
https://schema.management.azure.com/schemas/2015-04-01/Microsoft.NotificationHubs.json#/resourceDefinitions/notificationHubs
https://schema.management.azure.com/schemas/2015-11-01/Microsoft.Network.json#/resourceDefinitions/trafficManagerProfiles
https://schema.management.azure.com/schemas/2017-03-01/Microsoft.Network.json#/resourceDefinitions/trafficManagerProfiles
https://schema.management.azure.com/schemas/2017-05-01/Microsoft.Network.json#/resourceDefinitions/trafficManagerProfiles
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts
https://schema.management.azure.com/schemas/2016-01-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts
https://schema.management.azure.com/schemas/2018-02-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_blobServices_containers_immutabilityPolicies
https://schema.management.azure.com/schemas/2018-07-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_blobServices
https://schema.management.azure.com/schemas/2018-07-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_blobServices_containers_immutabilityPolicies
https://schema.management.azure.com/schemas/2018-11-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_blobServices
https://schema.management.azure.com/schemas/2018-11-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_blobServices_containers_immutabilityPolicies
https://schema.management.azure.com/schemas/2018-11-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_managementPolicies
https://schema.management.azure.com/schemas/2019-04-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_blobServices
https://schema.management.azure.com/schemas/2019-04-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_blobServices_containers_immutabilityPolicies
https://schema.management.azure.com/schemas/2019-04-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_managementPolicies
https://schema.management.azure.com/schemas/2019-06-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_blobServices
https://schema.management.azure.com/schemas/2019-06-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_blobServices_containers_immutabilityPolicies
https://schema.management.azure.com/schemas/2019-06-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_fileServices
https://schema.management.azure.com/schemas/2019-06-01/Microsoft.Storage.json#/resourceDefinitions/storageAccounts_managementPolicies
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.Compute.json#/resourceDefinitions/availabilitySets
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.Compute.json#/resourceDefinitions/extensions
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.Compute.json#/resourceDefinitions/virtualMachineScaleSets
https://schema.management.azure.com/schemas/2015-06-01/Microsoft.KeyVault.json#/resourceDefinitions/vaults
https://schema.management.azure.com/schemas/2014-06-01/Microsoft.Web.json#/resourceDefinitions/sites
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.Web.json#/resourceDefinitions/sites
https://schema.management.azure.com/schemas/2015-04-01/Microsoft.NotificationHubs.json#/resourceDefinitions/namespaces
https://schema.management.azure.com/schemas/2015-08-01/Microsoft.Compute.json#/resourceDefinitions/virtualMachines
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones_A
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones_AAAA
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones_CNAME
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones_MX
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones_NS
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones_PTR
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones_SOA
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones_SRV
https://schema.management.azure.com/schemas/2016-04-01/Microsoft.Network.json#/resourceDefinitions/dnszones_TXT
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Logic.json#/resourceDefinitions/workflows
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Logic.json#/resourceDefinitions/integrationAccounts
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Logic.json#/resourceDefinitions/integrationAccounts_agreements
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Logic.json#/resourceDefinitions/integrationAccounts_certificates
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Logic.json#/resourceDefinitions/integrationAccounts_maps
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Logic.json#/resourceDefinitions/integrationAccounts_partners
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Logic.json#/resourceDefinitions/integrationAccounts_schemas
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Logic.json#/resourceDefinitions/integrationAccounts_assemblies
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Logic.json#/resourceDefinitions/integrationAccounts_batchConfigurations
https://schema.management.azure.com/schemas/2016-10-01/Microsoft.Logic.json#/resourceDefinitions/workflows
https://schema.management.azure.com/schemas/2017-07-01/Microsoft.Logic.json#/resourceDefinitions/workflows
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Web.json#/resourceDefinitions/connections
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Web.json#/resourceDefinitions/connectionGateways
https://schema.management.azure.com/schemas/2016-06-01/Microsoft.Web.json#/resourceDefinitions/customApis
https://schema.management.azure.com/schemas/2015-10-01/Microsoft.Media.json#/resourceDefinitions/mediaServices
https://schema.management.azure.com/schemas/2016-03-01/Microsoft.ServiceFabric.json#/resourceDefinitions/clusters
https://schema.management.azure.com/schemas/2016-09-01/Microsoft.Authorization.json#/resourceDefinitions/locks
https://schema.management.azure.com/schemas/2017-03-01/Microsoft.ApiManagement.json#/resourceDefinitions/service_identityProviders
https://schema.management.azure.com/schemas/2017-03-01/Microsoft.ApiManagement.json#/resourceDefinitions/service_notifications
https://schema.management.azure.com/schemas/2017-03-01/Microsoft.ApiManagement.json#/resourceDefinitions/service_templates
https://schema.management.azure.com/schemas/2018-01-01/Microsoft.ApiManagement.json#/resourceDefinitions/service_identityProviders
https://schema.management.azure.com/schemas/2018-01-01/Microsoft.ApiManagement.json#/resourceDefinitions/service_notifications
https://schema.management.azure.com/schemas/2018-01-01/Microsoft.ApiManagement.json#/resourceDefinitions/service_templates
https://schema.management.azure.com/schemas/2019-01-01/Microsoft.ApiManagement.json#/resourceDefinitions/service_identityProviders
https://schema.management.azure.com/schemas/2019-01-01/Microsoft.ApiManagement.json#/resourceDefinitions/service_notifications
https://schema.management.azure.com/schemas/2019-01-01/Microsoft.ApiManagement.json#/resourceDefinitions/service_templates
https://schema.management.azure.com/schemas/2017-03-30/Microsoft.Compute.json#/resourceDefinitions/extensions
https://schema.management.azure.com/schemas/2017-04-01/Microsoft.Insights.json#/resourceDefinitions/actionGroups
https://schema.management.azure.com/schemas/2017-04-01/Microsoft.Insights.json#/resourceDefinitions/activityLogAlerts
https://schema.management.azure.com/schemas/2017-11-15/Microsoft.TimeSeriesInsights.json#/resourceDefinitions/environments_eventSources
https://schema.management.azure.com/schemas/2017-12-01/Microsoft.Compute.json#/resourceDefinitions/extensions
https://schema.management.azure.com/schemas/2017-12-01/Microsoft.Compute.json#/resourceDefinitions/vmssExtensions
https://schema.management.azure.com/schemas/2017-10-01/Microsoft.Insights.json#/resourceDefinitions/components_pricingPlans
https://schema.management.azure.com/schemas/2016-12-01/Microsoft.RecoveryServices.json#/resourceDefinitions/vaults_backupstorageconfig
https://schema.management.azure.com/schemas/2018-06-01/Microsoft.Security.json#/resourceDefinitions/pricings
https://schema.management.azure.com/schemas/2019-03-01/Microsoft.Insights.json#/resourceDefinitions/actionGroups
https://schema.management.azure.com/schemas/2018-02-01/Microsoft.Web.json#/resourceDefinitions/sites_config
https://schema.management.azure.com/schemas/2018-02-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_config
https://schema.management.azure.com/schemas/2018-11-01/Microsoft.Web.json#/resourceDefinitions/sites_config
https://schema.management.azure.com/schemas/2018-11-01/Microsoft.Web.json#/resourceDefinitions/sites_slots_config

Resource is defined in referenced spec, but is not actually referenced by 2015-01-01 main spec:

Sorry for the slightly wonky formatting of the below. The way to read it is as follows: {microsoft.network/v20170901}/RouteTablesRoutes means that there is a resource https://schema.management.azure.com/schemas/2017-09-01/Microsoft.Network.json#/resourceDefinitions/routeTablesRoutes (or in those case routeTables_Routes - we strip underscores) whose reference is missing from the 2015-01-01 main spec.

{microsoft.dbformariadb/v20180601}/ServersPrivateEndpointConnections
{microsoft.network/v20170901}/RouteTablesRoutes
{microsoft.devices/v20180401}/IotHubsCertificates
{microsoft.network/v20180101}/RouteTablesRoutes
{microsoft.network/v20181001}/ExpressRouteCrossConnectionsPeerings
{microsoft.network/v20181201}/NetworkInterfacesTapConfigurations
{microsoft.network/v20171001}/RouteTablesRoutes
{microsoft.apimanagement/v20160707}/ServiceApis
{microsoft.apimanagement/v20160707}/ServiceSubscriptions
{microsoft.apimanagement/v20160707}/ServiceProducts
{microsoft.apimanagement/v20160707}/ServiceGroups
{microsoft.apimanagement/v20160707}/ServiceCertificates
{microsoft.apimanagement/v20160707}/ServiceUsers
{microsoft.apimanagement/v20160707}/ServiceAuthorizationServers
{microsoft.apimanagement/v20160707}/ServiceLoggers
{microsoft.apimanagement/v20160707}/ServiceProperties
{microsoft.apimanagement/v20160707}/ServiceOpenidConnectProviders
{microsoft.apimanagement/v20160707}/ServiceBackends
{microsoft.apimanagement/v20160707}/ServiceIdentityProviders
{microsoft.network/v20171001}/NetworkSecurityGroupsSecurityRules
{microsoft.scheduler/v20160101}/JobCollectionsJobs
{microsoft.datalakestore/v20161101}/AccountsVirtualNetworkRules
{microsoft.compute/v20181001}/VirtualMachineScaleSetsVirtualmachines
{microsoft.compute/v20181001}/VirtualMachineScaleSetsExtensions
{microsoft.devtestlab/v20160515}/LabsCosts
{microsoft.devtestlab/v20160515}/LabsNotificationchannels
{microsoft.devtestlab/v20160515}/LabsServicerunners
{microsoft.devtestlab/v20160515}/LabsUsers
{microsoft.web/v20160801}/SitesSlotsAppsettings
{microsoft.web/v20160801}/Connectionstrings
{microsoft.web/v20160801}/Config
{microsoft.cognitiveservices/v20170418}/AccountsPrivateEndpointConnections
{microsoft.apimanagement/v20190101}/ServiceApisIssues
{microsoft.network/v20180801}/NetworkSecurityGroupsSecurityRules
{microsoft.documentdb/v20150408}/DatabaseAccountsApisKeyspacesSettings
{microsoft.network/v20190201}/NetworkInterfacesTapConfigurations
{microsoft.network/v20181001}/LoadBalancersInboundNatRules
{microsoft.devices/v20180122}/IotHubsCertificates
{microsoft.apimanagement/v20170301}/ServiceApiVersionSets
{microsoft.apimanagement/v20170301}/ServiceApisDiagnostics
{microsoft.apimanagement/v20170301}/ServiceApisIssues
{microsoft.network/v20190201}/RouteTablesRoutes
{microsoft.network/v20180201}/RouteTablesRoutes
{microsoft.network/v20171101}/NetworkSecurityGroupsSecurityRules
{microsoft.network/v20171101}/LoadBalancersInboundNatRules
{microsoft.network/v20190201}/LoadBalancersInboundNatRules
{microsoft.network/v20181201}/RouteTablesRoutes
{microsoft.network/v20180801}/RouteTablesRoutes
{microsoft.documentdb/v20150408}/DatabaseAccountsApisKeyspacesTablesSettings
{microsoft.compute/v20180601}/VirtualMachineScaleSetsVirtualmachines
{microsoft.compute/v20180601}/VirtualMachineScaleSetsExtensions
{microsoft.network/v20180201}/LoadBalancersInboundNatRules
{microsoft.network/v20180101}/LoadBalancersInboundNatRules
{microsoft.network/v20170901}/NetworkSecurityGroupsSecurityRules
{microsoft.documentdb/v20150408}/DatabaseAccountsApisDatabasesGraphsSettings
{microsoft.network/v20181201}/ExpressRouteCrossConnectionsPeerings
{microsoft.network/v20190201}/ExpressRouteCircuitsPeerings
{microsoft.network/v20170601}/NetworkSecurityGroupsSecurityRules
{microsoft.network/v20181101}/RouteTablesRoutes
{microsoft.network/v20180801}/LoadBalancersInboundNatRules
{microsoft.network/v20181001}/NetworkInterfacesTapConfigurations
{microsoft.network/v20181001}/ExpressRouteCircuitsPeerings
{microsoft.network/v20181201}/NetworkSecurityGroupsSecurityRules
{microsoft.devices/v20170701}/IotHubsCertificates
{microsoft.visualstudio/v20140226}/Project
{microsoft.network/v20180101}/NetworkSecurityGroupsSecurityRules
{microsoft.network/v20180201}/ExpressRouteCircuitsPeerings
{microsoft.network/v20180201}/ExpressRouteCircuitsAuthorizations
{microsoft.network/v20181001}/NetworkSecurityGroupsSecurityRules
{microsoft.network/v20170801}/RouteTablesRoutes
{microsoft.network/v20190201}/NetworkSecurityGroupsSecurityRules
{microsoft.network/v20181101}/LoadBalancersInboundNatRules
{microsoft.network/v20171101}/RouteTablesRoutes
{microsoft.network/v20180801}/NetworkInterfacesTapConfigurations
{microsoft.devices.provisioning/v20171115}/ProvisioningServicesCertificates
{microsoft.devtestlab/v20160515}/LabsVirtualmachinesSchedules
{microsoft.compute/v20190301}/VirtualMachinesExtensions
{microsoft.network/v20171001}/LoadBalancersInboundNatRules
{microsoft.apimanagement/v20180101}/ServiceApiVersionSets
{microsoft.compute/v20181001}/VirtualMachinesExtensions
{microsoft.documentdb/v20150408}/DatabaseAccountsApisDatabasesCollectionsSettings
{microsoft.apimanagement/v20180101}/ServiceApisDiagnostics
{microsoft.apimanagement/v20180101}/ServiceApisIssues
{microsoft.storsimple.1200/v20161001}/ManagersStorageDomains
{microsoft.network/v20180801}/ExpressRouteCircuitsPeerings
{microsoft.network/v20181201}/LoadBalancersInboundNatRules
{microsoft.network/v20170601}/RouteTablesRoutes
{microsoft.compute/v20190701}/VirtualMachinesExtensions
{microsoft.network/v20170901}/LoadBalancersInboundNatRules
{microsoft.documentdb/v20150408}/DatabaseAccountsApisDatabasesContainersSettings
{microsoft.network/v20180201}/NetworkSecurityGroupsSecurityRules
{microsoft.compute/v20190701}/VirtualMachineScaleSetsExtensions
{microsoft.network/v20170801}/LoadBalancersInboundNatRules
{microsoft.web/v20160801}/SitesAppsettings
{microsoft.web/v20160801}/Connectionstrings
{microsoft.web/v20160801}/Config
{microsoft.network/v20170801}/NetworkSecurityGroupsSecurityRules
{microsoft.network/v20181201}/ExpressRouteCircuitsPeerings
{microsoft.keyvault/v20150601}/SecretsChild
{microsoft.compute/v20180601}/VirtualMachinesExtensions
{microsoft.network/v20181001}/RouteTablesRoutes
{microsoft.cache/v20160401}/RedisFirewallRules
{microsoft.cache/v20160401}/RedisPatchSchedules
{microsoft.compute/v20190301}/VirtualMachineScaleSetsExtensions
{microsoft.documentdb/v20150408}/DatabaseAccountsApisTablesSettings
{microsoft.network/v20181101}/ExpressRouteCircuitsPeerings
{microsoft.network/v20181101}/NetworkInterfacesTapConfigurations
{microsoft.network/v20180201}/ExpressRouteCrossConnectionsPeerings
{microsoft.network/v20181101}/NetworkSecurityGroupsSecurityRules
{microsoft.dbforpostgresql/v20171201}/ServersAdministrators
{microsoft.network/v20180801}/ExpressRouteCrossConnectionsPeerings
{microsoft.network/v20190201}/ExpressRouteCrossConnectionsPeerings
{microsoft.network/v20181101}/ExpressRouteCrossConnectionsPeerings
azcloudfarmer commented 4 years ago

Hi @matthchr thanks for submitting the PR! We will be working on the 'name' fixes soon. Expect the fix to be submitted around ETA mid-August.

azcloudfarmer commented 4 years ago

Hi @matthchr quick update. For the 'name' fixes, we expect those to be resolved for the majority of the RPs on the list you provided, once their newly auto-generated schema gets merged (we are waiting for a review from those RPs). Can you please share with us the tool used to identify the missing required name field?

matthchr commented 4 years ago

@apclouds - The tool I used is actually a custom code generation tool which my team is working on. It's purpose isn't really to find these issues it just happens to have done so.

It seems like unless there won't be any handcrafted resources in this repo that it would make sense to have a basic validator tool that just walked all the resources and had some standard assertions.

The list that I've got in my head (and this may not be correct), is something like... When any spec is added or updated:

  1. All resources (defined in resourceDefinitions section) are referenced in all of the rollup specs (so 2015-01-01, 2019-xx-xx, etc etc).
  2. Each resource must have a type property, of type enum, with a single value.
  3. Each resource must have a name property, of type string.
  4. Each ref must lead to a definition that actually exists (both from the main 2015-01-01 rollup spec, as well as within a given document).
  5. If a resource is defined as a _childResource in a resources section, there is a corresponding resource with the same type in resourceDefinitions. Otherwise if an item in the resource section isn't marked as _childResource it should be pointing to an actual resource (#/resourceDefinitions/myResource for example)

There are probably more assertions that could be made with such a tool, like that resources are marking themselves as inheriting from resourceBase (I can't tell if this is supposed to be done or not, but it seems like it's supposed to?)

You could also flag suspiciously formatted specs, such as any that make use of anyOf (because anyOf is extremely weird and basically no deserialization technologies really support it well - people often use anyOf when they really mean allOf or oneOf).

Our tool is written in Go (whose JSON schema parsing is not all that friendly). I would imagine there are probably some good javascript/typescript libraries which could do all of the above quite easily though.

ghost commented 4 years ago

Hello @armleads-azure, @apclouds, @anthony-c-martin, @jorgecotillo! It looks like there is a schemas issue that needs your attention. Please investigate and confirm it is on your end. Thanks :smile:

alex-frankel commented 3 years ago

@anthony-c-martin / @apclouds - is this resolved in the 2019 root schema?

anthony-c-martin commented 3 years ago

The list that I've got in my head (and this may not be correct), is something like... When any spec is added or updated:

@matthchr Just to address these comments/questions:

  1. All resources (defined in resourceDefinitions section) are referenced in all of the rollup specs (so 2015-01-01, 2019-xx-xx, etc etc).

For performance reasons (partial pattern matching on the giant resources array is EXTREMELY expensive in JSON schema 4), the decision was made to stop adding new types to the older root schemas (2015-01-01 & 2014-04-01-preview). As such, 2019-04-01 is the only resourceGroup-level root schema that should contain all resource types. I'm working on updating the README to clarify this.

  1. Each resource must have a type property, of type enum, with a single value.
  2. Each resource must have a name property, of type string.

Technically in JSON schema 4, unless "additionalProperties": false is specified, every object type is open for extension. I agree however as a best practice, we should validate name & type.

  1. Each ref must lead to a definition that actually exists (both from the main 2015-01-01 rollup spec, as well as within a given document).

Do you have examples for where this is not the case? I thought we were validating this in our test suite, but if not, we definitely should be!

  1. If a resource is defined as a _childResource in a resources section, there is a corresponding resource with the same type in resourceDefinitions. Otherwise if an item in the resource section isn't marked as _childResource it should be pointing to an actual resource (#/resourceDefinitions/myResource for example)

I don't think I fully understanding this one - do you have an example?

anthony-c-martin commented 3 years ago

@anthony-c-martin / @apclouds - is this resolved in the 2019 root schema?

This issue is tracking a few different items. The lack of support for certain resource types is resolved in the 2019-04-01 schema, but we need to add a clarifying note to the README to explain this. The other items are common to all root schemas.

matthchr commented 3 years ago

Do you have examples for where this is not the case? I thought we were validating this in our test suite, but if not, we definitely should be!

I don't have any examples now -- I do think we saw it once but it's been long enough ago that I don't remember what it was. I think for this part you can table it for now if you think you're validating it and we can always re-raise an issue if we find another example of it.

If a resource is defined as a _childResource in a resources section, there is a corresponding resource with the same type in resourceDefinitions. Otherwise if an item in the resource section isn't marked as _childResource it should be pointing to an actual resource (#/resourceDefinitions/myResource for example)

I don't think I fully understanding this one - do you have an example?

For example looking at the Microsoft.Web 2016-08-01 API, there is a resource "sites_slot", which has the following:

        "resources": {
          "type": "array",
          "items": {
            "oneOf": [
              {
                "$ref": "#/definitions/sites_slots_virtualNetworkConnections_childResource"
              },
              {
                "$ref": "#/definitions/sites_slots_premieraddons_childResource"
              },
              {
                "$ref": "#/definitions/sites_slots_hybridconnection_childResource"
              },
              {
                "$ref": "#/definitions/sites_slots_hostNameBindings_childResource"
              },
              {
                "$ref": "#/definitions/sites_slots_domainOwnershipIdentifiers_childResource"
              },
              {
                "$ref": "#/definitions/sites_slots_deployments_childResource"
              },
              {
                "$ref": "#/definitions/sites_slots_appsettings_childResource"
              },
              {
                "$ref": "#/definitions/sites_slots_connectionstrings_childResource"
              },
              {
                "$ref": "#/definitions/sites_slots_config_childResource"
              },
              {
                "$ref": "#/definitions/sites_slots_publicCertificates_childResource"
              }
            ]
          }
        }

Each of these refs exist, but my expectation is that if something is a _childResource that there should be an actual resource (in the resources section) which corresponds to it. In this particular schema, there's a reference to sites_slots_connectionstrings_childResource, which looks like this:

    "sites_slots_connectionstrings_childResource": {
      "allOf": [
        {
          "$ref": "#/definitions/connectionstrings_childResource"
        }
      ],
      "description": "Microsoft.Web/sites/slots/config"
    },

connectionstrings_childResource certainly looks like a resource, but it's in the definitions section:

    "connectionstrings_childResource": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string",
          "enum": [
            "connectionstrings"
          ]
        },
        "type": {
          "type": "string",
          "enum": [
            "config"
          ]
        },
        "apiVersion": {
          "type": "string",
          "enum": [
            "2016-08-01"
          ]
        },
        "properties": {
          "oneOf": [
            {
              "type": "object",
              "additionalProperties": {
                "oneOf": [
                  {
                    "$ref": "#/definitions/ConnectionString_properties"
                  },
                  {
                    "$ref": "https://schema.management.azure.com/schemas/common/definitions.json#/definitions/expression"
                  }
                ]
              }
            },
            {
              "$ref": "https://schema.management.azure.com/schemas/common/definitions.json#/definitions/expression"
            }
          ],
          "description": "ARM envelope properties"
        }
      },
      "required": [
        "type",
        "apiVersion",
        "properties"
      ]
    },

As far as I can tell, there's no actual resource for a connectionString in the resources section. The above quote (my item 5) was me trying to state: "If there's a thing_childresource then the implication is that there is a thing which is a resource in the resources section.

Note: The above is just an example, there are other occurrences of this problem (if it is indeed a problem) in other specs.