dry-rb / dry-schema

Coercion and validation for data structures
https://dry-rb.org/gems/dry-schema
MIT License
422 stars 110 forks source link

Simple presence validation too slow #339

Open xronos-i-am opened 3 years ago

xronos-i-am commented 3 years ago

Describe the bug

Documentation says the dry-shema multiple times faster than ActiveRecord/ActiveModel::Validations. But it is not true for simple presence/filled validation. Сorrect me if I'm wrong please

To Reproduce

require 'bundler/inline'

gemfile do
  gem "activemodel"
  gem "dry-schema"
  gem "benchmark-ips"
end

require "benchmark/ips"
require "active_model"

class Form
  include ActiveModel::Model
  include ActiveModel::Attributes

  attribute :account

  validates :account, presence: true
end

Schema = Dry::Schema.Params do
  required(:account).filled
end

form = Form.new(account: '10')

Benchmark.ips do |x|
  x.report('Form') do
    form.valid?
    form.errors.clear
  end
  x.report('Schema') do
    Schema.call(account: '10').success?
  end
  x.compare!
end

# Warming up --------------------------------------
#                 Form    10.644k i/100ms
#               Schema     3.751k i/100ms
# Calculating -------------------------------------
#                 Form    108.634k (± 4.6%) i/s -    542.844k in   5.009453s
#               Schema     36.554k (± 6.0%) i/s -    183.799k in   5.050119s

# Comparison:
#                 Form:   108633.9 i/s
#               Schema:    36554.3 i/s - 2.97x  (± 0.00) slower

Expected behavior

The most common validation case should be faster. Maybe we can do a quick return from Value.call for simple cases.

My environment

solnic commented 3 years ago

Thanks for reporting this. Benchmarking is hard though - they way you wrote it makes it much faster in case of AM because you initialized the form just once, in reality, you need to instantiate it every time because that's realistic.

With this:

require "benchmark/ips"
require "active_model"
require "dry/schema"

class Form
  include ActiveModel::Model
  include ActiveModel::Attributes

  attribute :account

  validates :account, presence: true
end

Schema = Dry::Schema.Params do
  required(:account).filled
end

Benchmark.ips do |x|
  x.report('Form') do
    form = Form.new(account: '10')
    form.valid?
    # form.errors.clear
  end
  x.report('Schema') do
    Schema.call(account: '10').success?
  end
  x.compare!
end

Performance on MRI 3.0.0 is actually the same, so AM must've become faster lately. I'll see how it compares in more complex examples and provide better info in the docs. Until then, I'll keep this issue opened.

xronos-i-am commented 3 years ago

I'v got the following results on 2.5:

Benchmark.ips do |x|
  x.report('Form') do
    Form.new(account: '10').valid?
  end
  x.report('Schema') do
    Schema.call(account: '10').success?
  end
  x.compare!
end

# Warming up --------------------------------------
#                 Form     4.852k i/100ms
#               Schema     3.438k i/100ms
# Calculating -------------------------------------
#                 Form     48.376k (± 4.3%) i/s -    242.600k in   5.025081s
#               Schema     39.966k (± 2.8%) i/s -    202.842k in   5.079528s

# Comparison:
#                 Form:    48375.6 i/s
#               Schema:    39966.0 i/s - 1.21x  (± 0.00) slower
flash-gordon commented 3 years ago

It's important to realize that comparing trivial cases is not really representative. It can very well be that any lib gets exponentially slower as rules grow complex.