hashicorp / terraform-provider-awscc

Terraform AWS Cloud Control provider
https://registry.terraform.io/providers/hashicorp/awscc/latest/docs
Mozilla Public License 2.0
249 stars 113 forks source link

Property pattern validation #88

Closed ewbankkit closed 2 years ago

ewbankkit commented 3 years ago

Relates #45. Relates https://github.com/hashicorp/aws-cloudformation-resource-schema-sdk-go/issues/7.

Due to regex syntax mismatches (e.g. negative lookahead) between the JSON Schema/ECMA-262 specification and the gojsonschema implementation which uses Go's re2, when downloading CloudFormation resource schemas we are currently rewriting all pattern (and propertyPattern) values to the empty string, which in effect means no validation takes place.

Longer term we should investigate less brute force workarounds.

PatMyron commented 2 years ago

related: https://github.com/aws-cloudformation/cloudformation-cli/pull/675#discussion_r566298712

PatMyron commented 2 years ago

will likely run into https://github.com/golang/go/issues/7252 too

grep -E '\d{4}}' *
aws-appintegrations-eventintegration.json:          "pattern" : "^arn:aws:[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$",
aws-appintegrations-eventintegration.json:      "pattern" : "^arn:aws:[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$",
aws-apprunner-service.json:          "pattern" : "arn:aws(-[\\w]+)*:[a-z0-9-\\\\.]{0,63}:[a-z0-9-\\\\.]{0,63}:[0-9]{12}:(\\w|\\/|-){1,1011}"
aws-apprunner-service.json:      "pattern" : "arn:aws(-[\\w]+)*:[a-z0-9-\\\\.]{0,63}:[a-z0-9-\\\\.]{0,63}:[0-9]{12}:(\\w|\\/|-){1,1011}"
aws-apprunner-service.json:      "pattern" : "arn:aws(-[\\w]+)*:[a-z0-9-\\\\.]{0,63}:[a-z0-9-\\\\.]{0,63}:[0-9]{12}:(\\w|\\/|-){1,1011}"
aws-chatbot-slackchannelconfiguration.json:      "pattern" : "^arn:(aws[a-zA-Z-]*)?:[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-chatbot-slackchannelconfiguration.json:        "pattern" : "^arn:(aws[a-zA-Z-]*)?:[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-chatbot-slackchannelconfiguration.json:      "pattern" : "^arn:(aws[a-zA-Z-]*)?:chatbot:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-chatbot-slackchannelconfiguration.json:        "pattern" : "^(^$|arn:aws:iam:[A-Za-z0-9_\\/.-]{0,63}:[A-Za-z0-9_\\/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_\\/+=,@.-]{0,1023})$"
aws-finspace-environment.json:          "pattern" : "^https?://[-a-zA-Z0-9+&@#/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#/%=~_|]{1,1000}"
aws-finspace-environment.json:          "pattern" : "^https?://[-a-zA-Z0-9+&@#/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#/%=~_|]{1,1000}"
aws-finspace-environment.json:      "pattern" : "^[a-zA-Z0-9. ]{1,1000}$"
aws-finspace-environment.json:      "pattern" : "^[-a-zA-Z0-9+&@#/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#/%=~_|]{1,1000}"
aws-finspace-environment.json:      "pattern" : "^[a-zA-Z-0-9-:\\/.]*{1,1000}$"
aws-finspace-environment.json:      "pattern" : "^[a-zA-Z-0-9-:\\/]*{1,1000}$"
aws-groundstation-config.json:      "pattern" : "^[{}\\[\\]:.,\"0-9A-z\\-_\\s]{1,8192}$"
aws-kendra-datasource.json:      "pattern" : "arn:[a-z0-9-\\.]{1,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[^/].{0,1023}"
aws-kendra-datasource.json:      "pattern" : "arn:[a-z0-9-\\.]{1,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[^/].{0,1023}"
aws-kendra-faq.json:      "pattern" : "arn:[a-z0-9-\\.]{1,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[^/].{0,1023}"
aws-kendra-index.json:      "pattern" : "arn:[a-z0-9-\\.]{1,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[^/].{0,1023}"
aws-lookoutequipment-inferencescheduler.json:          "pattern" : "^[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,2048}$",
aws-lookoutequipment-inferencescheduler.json:      "pattern" : "^[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,2048}$",
aws-resiliencehub-app.json:      "pattern" : "^arn:(aws|aws-cn|aws-iso|aws-iso-[a-z]{1}|aws-us-gov):[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:([a-z]{2}-((iso[a-z]{0,1}-)|(gov-)){0,1}[a-z]+-[0-9]):[0-9]{12}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-resiliencehub-app.json:      "pattern" : "^arn:(aws|aws-cn|aws-iso|aws-iso-[a-z]{1}|aws-us-gov):[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:([a-z]{2}-((iso[a-z]{0,1}-)|(gov-)){0,1}[a-z]+-[0-9]):[0-9]{12}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-resiliencehub-resiliencypolicy.json:      "pattern" : "^arn:(aws|aws-cn|aws-iso|aws-iso-[a-z]{1}|aws-us-gov):[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:([a-z]{2}-((iso[a-z]{0,1}-)|(gov-)){0,1}[a-z]+-[0-9]):[0-9]{12}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-wafv2-rulegroup.json:      "pattern" : "^[0-9A-Za-z_:-]{1,1024}$"
aws-wafv2-rulegroup.json:      "pattern" : "^[0-9A-Za-z_:-]{1,1024}$"
aws-wafv2-webacl.json:      "pattern" : "^[0-9A-Za-z_:-]{1,1024}$"
aws-wafv2-webacl.json:      "pattern" : "^[0-9A-Za-z_:-]{1,1024}$"
ewbankkit commented 2 years ago

Until either

  1. Go's regexp package supports ECMA-262 pattern
  2. All CloudFormation resource schemas use a common subset of supported patterns

we could make some progress by checking during schema download whether a pattern is a valid Go regexp pattern. This would involve changes to https://github.com/hashicorp/aws-cloudformation-resource-schema-sdk-go/blob/main/sanitize.go.