Generating errors for unexpected keys in Hash

[backus]

I would love to be able to define a schema which generated errors if it was given unexpected keys:

require 'dry/validation'

schema = Dry::Validation.Schema(strict: true) do
  hash? do
    required(:foo).filled(:int?)
    required(:bar).filled(:str?)
  end
end

outcome = schema.call(foo: 1, bar: "two", baz: 3.0)
p outcome.messages # => {:baz=>["is not allowed"]}

This is really important for defining a public JSON API where you want to reject invalid user input.

[backus]

Here is an abridged discussion of this from the gitter chat:

John Backus @backus May 26 12:05

Is there any way to generate an error if a hash has an unexpected key with dry-validation?

Piotr Solnica @solnic May 26 12:53

no, not yet

Piotr Solnica @solnic May 26 12:53

I mean there's no built-in predicate for that

John Backus @backus May 26 12:54

Any way to define it with a custom predicate? My impression was that you weren't really able to define validations for the entire object being passed to the validator

Piotr Solnica @solnic May 26 13:09

it is possible but with limitations, so you can use hash? do ... end in the root this will be applied to the input that's passed to validation schema we can add support for more

Andy Holland @AMHOL May 26 13:10

I ended up with something like:

gemfile(true) { gem 'dry-validation', github: 'dry-rb/dry-validation' }

module MyPredicates
  include Dry::Logic::Predicates

  ALLOWED_KEYS = %i(
    id
    name
    email
  ).freeze

  predicate(:restricted_hash?) do |hash|
    ALLOWED_KEYS & hash.keys == ALLOWED_KEYS
  end
end

schema = Dry::Validation.Schema do
  configure do
    config.predicates = MyPredicates
  end

  restricted_hash? do
    required(:id).filled(:int?)
    required(:name).filled
    required(:email).filled
  end
end

schema.(id: 1, name: 'Joe', email: 'joe@hotmail.com')

That seems to work, if you add the error message

[solnic]

We could now easily build this on top of input macro, although I gotta improve it first as the way I implemented it turned out to be problematic when nested schemas are used and error messages are messed up a bit (see #200). Then it's only a matter of extending input to accept more than one predicate and adding a new predicate like restricted_hash? which can just look at rules.keys in schema and that's it.

[qwert321]

Thanks. This is a MUST if dry-validation want to replace strong parameters in rails. This is a protection from massive assignment in rails.

[fledman]

I don't know how kosher this is, but you can make a custom predicate that dynamically looks up the schema keys, and attach it to the input macro:

require 'dry-validation'

class Base < Dry::Validation::Schema
  def strict_keys?(input)
    (input.keys - self.rules.keys).empty?
  end

  def self.messages
    super.merge(en: { errors: { strict_keys?: 'has unknown keys' } } )
  end
end

schema = Dry::Validation.Schema(Base) do
  input :hash?, :strict_keys?
  required(:foo).filled(:int?)
  optional(:bar).filled(:str?)
end

schema.call(foo: 1, bar: 2)
# => #<Dry::Validation::Result output={:foo=>1, :bar=>2} errors={:bar=>["must be a string"]}>

schema.call(foo: 1, bar: "2")
# => #<Dry::Validation::Result output={:foo=>1, :bar=>"2"} errors={}> 

schema.call(foo: 1, bar: "2", baz: 3)
# => #<Dry::Validation::Result output={:foo=>1, :bar=>"2", :baz=>3} errors=["has unknown keys"]>

schema.call(foo: 1, bar: 2, baz: 3)
# => #<Dry::Validation::Result output={:foo=>1, :bar=>2, :baz=>3} errors=["has unknown keys"]>

the one downside I noticed is that if the input rule fails, it doesn't check the rest of the rules

[fledman]

as an aside, it would be nice if there was a way to pass data from the predicate into the error message e.g. has unknown keys: ["baz"]

[solnic]

This will be supported in 1.0

[pascalbetz]

Is there any news on this or should I go with @fledman proposal?

I want to validate parameters that are passed in through an API before I send them to the next system. So i thougt i'd would be nice if I tell the client if he sends params that I don't know instead of just ignoring them with

configure do
  config.input_processor = :sanitizer
end

It's unlikely that unknown parameters are sent, so I'd be OK with not checking the rest of the rules.

Thanks dry-rb team for the good work.

[fledman]

@pascalbetz you can hack together a fake check (what the high level rules use) which prevents early stopping and adds a specific error message for each unknown key.

But it is super dirty & highly dependent on the current internal api.

expand for terrible hack

```ruby require 'dry-validation' module StrictKeys def self.name :__strict_keys__ end def self.to_ast [:type, self] end def self.rule @rule ||= Checker.new end def self.failure(key) Dry::Logic::Result.new(false, key) do [:key, [key, [:predicate, [name, []]]]] end end class Checker < Dry::Validation::Guard def initialize; end def with(*); self; end def call(input, results) input.each do |k,v| results[k] = StrictKeys.failure(k) unless results.key?(k) end nil end end end class Base < Dry::Validation::Schema def self.messages super.merge(en: { errors: { StrictKeys.name => 'is an unknown key' } } ) end end schema = Dry::Validation.Schema(Base) do checks << StrictKeys required(:foo).filled(:int?) optional(:bar).filled(:str?) end ```

[pascalbetz]

@fledman thanks for the explanation. I'm in the process of refactoring the APIs and see if/how I can make use of your proposal.

[sirion1987]

I found this solution: https://github.com/sirion1987/freckle-io/blob/master/lib/freckle_io/validator/user.rb and https://github.com/sirion1987/freckle-io/blob/master/lib/freckle_io/validator/restricted_hash.rb.

[joelvh]

Will Schema#strict be implemented in dry-validation v1.0.0 Contract as a feature, or do we need to key into the underlying schema?

[solnic]

@joelvh I'd prefer to have it as part of dry-schema but we'll see what makes more sense.

[joelvh]

@solnic that was quick! I just realized that Schema#strict is in dry-types, which dry-schema and dry-validation don't (?) use under the hood?

Lots of things to reconcile getting things upgraded to v1.0.0 - happy to see many of the changes!

[solnic]

@solnic that was quick!

You commented while I was going through my inbox 😄

I just realized that Schema#strict is in dry-types, which dry-schema and dry-validation don't (?) use under the hood?

Strict hash schemas from dry-types are not used at all by dry-schema/validation. In order to validate input's keys, we need support for "base" errors. This was added to dry-validation so now dry-schema needs to catch up.

Lots of things to reconcile getting things upgraded to v1.0.0 - happy to see many of the changes!

Yes it's a big step forward with dry-types/logic/schema/validation reaching 1.0.0 😄

[jacek213]

Still not supported in 1.4?

[solnic]

@jacek213 yes, it's not yet supported