Closed nnslvp closed 2 years ago
@yahorbukhta Thanks for pointing out, will check it on Ruby 3
yes, after test, fast_jsonparser became slow in ruby 3. some benchmark results:
in ruby 3.0.1
-----------------------------------load file------------------------------------
user system total real
json load_file: 1.274003 0.080859 1.354862 ( 1.473138)
oj load_file: 1.246956 0.149348 1.396304 ( 2.428900)
fast_jsonparser load_file: 2.428389 0.057700 2.486089 ( 2.568472)
-------------------------load file with symbolize_keys--------------------------
user system total real
json load_file: 1.164803 0.029896 1.194699 ( 1.293561)
oj load_file: 1.299518 0.104492 1.404010 ( 2.417975)
fast_jsonparser load_file: 2.529584 0.000000 2.529584 ( 2.618218)
-----------------------------------parse str------------------------------------
user system total real
json parse: 1.231608 0.000000 1.231608 ( 1.289384)
oj load: 1.032108 0.000000 1.032108 ( 1.044259)
fast_jsonparser parse: 2.473172 0.019797 2.492969 ( 2.502798)
-------------------------parse str with symbolize_keys--------------------------
user system total real
json parse: 1.284523 0.000000 1.284523 ( 1.296915)
oj load: 1.356738 0.000000 1.356738 ( 1.369711)
fast_jsonparser parse: 2.611612 0.006931 2.618543 ( 2.630617)
in ruby 2.7.1
-----------------------------------load file------------------------------------
user system total real
json load_file: 1.549832 0.054677 1.604509 ( 1.691528)
oj load_file: 1.341627 0.027201 1.368828 ( 2.379786)
fast_jsonparser load_file: 0.742486 0.041506 0.783992 ( 0.873495)
-------------------------load file with symbolize_keys--------------------------
user system total real
json load_file: 1.216889 0.018728 1.235617 ( 1.331226)
oj load_file: 1.402029 0.024825 1.426854 ( 2.442827)
fast_jsonparser load_file: 0.704549 0.000000 0.704549 ( 0.793270)
-----------------------------------parse str------------------------------------
user system total real
json parse: 1.339900 0.000000 1.339900 ( 1.346746)
oj load: 1.092986 0.000000 1.092986 ( 1.099830)
fast_jsonparser parse: 0.698173 0.000000 0.698173 ( 0.704163)
-------------------------parse str with symbolize_keys--------------------------
user system total real
json parse: 1.176147 0.000000 1.176147 ( 1.184036)
oj load: 1.162656 0.000000 1.162656 ( 1.169610)
fast_jsonparser parse: 0.692087 0.000000 0.692087 ( 0.698028)
FYI this is still the case - I'm also on M1. Is there a fix coming?
$ ruby -v
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [arm64-darwin21]
require 'fast_jsonparser'
require 'json'
require 'benchmark'
require_relative './lib/helpers'
json_files = get_auction_files(true)
puts "File count: #{json_files.size}"
Benchmark.bm(100) do |x|
x.report("(warm up)") do
json_files.each {|f| File.read(f) }
end
x.report("FastJsonparser.load(f)") do
json_files.each {|f| FastJsonparser.load(f) }
end
x.report("JSON.parse(File.read(f))") do
json_files.each {|f| JSON.parse(File.read(f)) }
end
end
File count: 65317
user system total real
(warm up) 0.203463 0.813508 1.016971 ( 1.017386)
FastJsonparser.load(f) 4.499778 0.858779 5.358557 ( 5.359655)
JSON.parse(File.read(f)) 0.648639 0.783460 1.432099 ( 1.432469)
Hi @ukd1 Can you confirm that solution mentioned in https://github.com/anilmaurya/fast_jsonparser/pull/22/files resolve this issue ?
@anilmaurya I don't have the time I'm afraid! They say it does in the PR https://github.com/saka1/simdjson_ruby/pull/24. Are you planning on fixing this?
Yes, will fix it in couple of days.
I am switching a project I'm on to use this libray, and I can't create the PR for it until this is fixed (or I'll have to commit again with a fixed library).
Any ETA on when we can get it merged? I hate to be a nag, but I'm blocked and you've already done the awesome work :)
@fearoffish Thanks for reminder.
Releasing new version in sometime.
Released version 0.6.0
with fix 🌟
BTW, retried this today, and it's still slow. I switched back to oj.
$ ruby -v
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23]
This solutions became slow in Ruby 3