adzap / timeliness

Fast date/time parsing for the control freak.
http://github.com/adzap/timeliness
MIT License
224 stars 27 forks source link

Higher than expected memory allocation #48

Open schneems opened 1 month ago

schneems commented 1 month ago

Thanks for the gem. I was looking for an alternative to Time that was faster and used less memory. I can't speak to the speed, but using the memory_profiler gem I found that this uses considerably more allocations than Time.parse.

Here's an example string that I'm parsing:

"2024-07-02T18:52:15Z"

Here's the memory_profiler top level allocations for the app I'm profiling, before with Time.parse:

Total allocated: 161056901 bytes (2007186 objects)
Total retained:  17832 bytes (130 objects)

After with Timeliness:

Total allocated: 399946421 bytes (3436459 objects)
Total retained:  6488 bytes (103 objects)

Reproduction

You can see this with an example script:

$ cat scratch.rb
require 'memory_profiler'
require 'timeliness'
require 'time'

STRING = "2024-07-02T18:52:15Z"

puts
puts "## Time.parse"
puts

report = MemoryProfiler.report do
  1000.times.each do
    Time.parse(STRING)
  end
end
report.pretty_print

puts
puts "## Timeliness:"
puts

report = MemoryProfiler.report do
  1000.times.each do
    Timeliness.parse(STRING)
  end
end
report.pretty_print

puts Timeliness.parse(STRING).class
⛄️ 3.3.1 🚀 /tmp/cefa184c397e89a3cc6c5761cc916e75
$ ruby scratch.rb | grep "Total allocated" -B2
## Time.parse

Total allocated: 2061656 bytes (29037 objects)
--
## Timeliness:

Total allocated: 7224520 bytes (55006 objects)

Thoughts

It seems that on the first allocation that Time.parse uses more memory, but on subsequent parses Timeliness.parse uses more. Perhaps there is some memorization trick that Time.parse is using to reduce memory that could also be used here.

adzap commented 1 month ago

Thanks for the report. I will look into it.

I haven't looked at the speed of the gem in a long time. It was faster in the early days of this gem for most common formats. But given all of the Ruby perf work in recent years, it's time I reviewed that.

These days I use it mainly for it's format flexibility. But a memory blow out is no good, so I'd be keen to fix that.

schneems commented 1 month ago

Cool, thanks for your work! It's not urgent for me, I'm not blocked. I just thought you might want to know 💜