xronos-i-am
commented
3 years ago to_json/from_json suggested to be replaced with Marshal methods _dump/self._load. Perfomance of Serializing/deserializing is decreased a little bit. But usage is become easier. It allows us to override serializer driver (see below) also
# frozen_string_literal: true
require 'bundler/inline'
gemfile do
gem 'mini_sql', github: 'discourse/mini_sql'
gem 'benchmark-ips'
end
require 'benchmark/ips'
require 'mini_sql'
require 'json'
class Serializer < Array
MAX_CACHE_SIZE = 500
def initialize(result)
replace(result)
end
def self.marshallable(result)
new(result)
end
def _dump(level)
JSON.generate(serialize)
end
def self._load(dump)
materialize(JSON.parse(dump))
end
private
def serialize
if length == 0
{}
else
{
"decorator" => first.class.decorator.to_s,
"fields" => first.to_h.keys,
"data" => map(&:values),
}
end
end
def self.materialize(wrapper)
if !wrapper["data"]
[]
else
materializer = cached_materializer(wrapper['fields'], wrapper['decorator'])
wrapper["data"].map do |row|
materializer.materialize(row)
end
end
end
def self.cached_materializer(fields, decorator_module = nil)
@cache ||= {}
key = fields
m = @cache.delete(key)
if m
@cache[key] = m
else
m = @cache[key] = materializer(fields)
@cache.shift if @cache.length > MAX_CACHE_SIZE
end
if decorator_module && decorator_module.length > 0
decorator = Kernel.const_get(decorator_module)
m = m.decorated(decorator)
end
m
end
def self.materializer(fields)
Class.new do
extend MiniSql::Decoratable
include MiniSql::Result
attr_accessor(*fields)
instance_eval <<~RUBY
def materialize(values)
r = self.new
#{col = -1; fields.map { |f| "r.#{f} = values[#{col += 1}]" }.join("; ")}
r
end
RUBY
end
end
end
PgResult =
Struct.new(:rows) do
def ntuples
rows.size
end
def fields
@fields ||= rows[0].keys
end
def getvalue(row_num, col_num)
rows[row_num][fields[col_num]]
end
end
module TopicDecorator
def permalink
"#{id}-#{title}"
end
end
topics =
10.times.map do |i|
id = rand(1..9999)
{
id: id,
title: "topic #{id}",
}
end
result = PgResult.new(topics)
materialized = MiniSql::Postgres::Connection.default_deserializer_cache.materialize(result, TopicDecorator)
# Benchmark.ips do |r|
# r.report("to_json") do
# Marshal.dump(MiniSql::Serializer.to_json(materialized))
# end
# r.report("marshal") do
# Marshal.dump(Serializer.marshallable(materialized))
# end
# r.compare!
# end
# Warming up --------------------------------------
# to_json 5.349k i/100ms
# marshal 4.632k i/100ms
# Calculating -------------------------------------
# to_json 51.900k (± 2.8%) i/s - 262.101k in 5.054218s
# marshal 47.426k (± 3.1%) i/s - 240.864k in 5.083753s
# Comparison:
# to_json: 51900.1 i/s
# marshal: 47426.2 i/s - 1.09x (± 0.00) slower
json = Marshal.dump(MiniSql::Serializer.to_json(materialized))
marshal = Marshal.dump(Serializer.marshallable(materialized))
Benchmark.ips do |r|
r.report("from_json") do
MiniSql::Serializer.from_json(Marshal.load(json))
end
r.report("marshal") do
Marshal.load(marshal)
end
r.compare!
end
# Warming up --------------------------------------
# from_json 5.586k i/100ms
# marshal 5.137k i/100ms
# Calculating -------------------------------------
# from_json 53.648k (± 2.3%) i/s - 273.714k in 5.104828s
# marshal 50.210k (± 2.1%) i/s - 251.713k in 5.015367s
# Comparison:
# from_json: 53648.0 i/s
# marshal: 50209.9 i/s - 1.07x (± 0.00) slower
eviltrout
SamSaffron
ermolaev
The PR has 2 goals: 1) Simplify usage of serializer with Rails.cache 2) Speed up deserialization
To fetch cached MiniSql results we have to do something like this:
With the proposed changes we can able to simplify code: