sparklemotion / nokogiri

Nokogiri (鋸) makes it easy and painless to work with XML and HTML from Ruby.
https://nokogiri.org/
MIT License
6.14k stars 897 forks source link

explore: optimize `Node#at_css` and `#at_xpath` #2213

Closed flavorjones closed 3 months ago

flavorjones commented 3 years ago

Currently, #at_css and #at_xpath execute the entire XPath query with multiple results, creates the NodeSet and wraps each result as a Ruby object before discarding all but the first result.

It should be possible to optimize this, both at the XPath layer and while marshalling results.

At the XPath layer, let's play with variations of (original-query)[1]

At the marshalling layer, let's discard the NodeSet and just return the single Ruby object.

flavorjones commented 3 months ago

Holy cow, I just discovered that libxml2 will automatically try to optimize any expression of the form (...)[1]:

#! /usr/bin/env ruby

require "bundler/inline"

gemfile do
  source "https://rubygems.org"
  gem "nokogiri", path: "."
  gem "benchmark-ips"
end

xml = "<root>" +
      (1..2000).map { |i| "<item>#{i}</item>" }.join +
      "</root>"
doc = Nokogiri::XML(xml)

Benchmark.ips do |x|
  x.report("optimized xpath") do
    result = doc.xpath("(//item)[1]")
    raise "unexpected result" unless result.size == 1
  end

  x.report("unoptimized xpath") do
    result = doc.xpath("//item[1]")
    raise "unexpected result" unless result.size == 1
  end

  x.compare!
end

reports

Warming up --------------------------------------
     optimized xpath     1.808k i/100ms
   unoptimized xpath     1.425k i/100ms
Calculating -------------------------------------
     optimized xpath     18.001k (± 1.6%) i/s -     90.400k in   5.023099s
   unoptimized xpath     13.702k (± 5.4%) i/s -     68.400k in   5.008044s

Comparison:
     optimized xpath:    18001.3 i/s
   unoptimized xpath:    13702.3 i/s - 1.31x  (± 0.00) slower

Seems like, other than having to deal with the crappy positional arguments for all the methods in Searchable, this should be an easy way to speed up at_css and at_xpath.

flavorjones commented 3 months ago

Hmm, actually this may not be as big of a win as I thought for some reason ...

xml = "<root>" +
      (1..2000).map { |i| "<item>#{i}</item>" }.join +
      "</root>"
doc = Nokogiri::XML(xml)

Benchmark.ips do |x|
  x.report("optimized xpath") do
    doc.xpath("(//item)[1]")
  end

  x.report("unoptimized xpath") do
    doc.at_xpath("//item")
  end

  x.compare!
end

reports

Warming up --------------------------------------
     optimized xpath     1.547k i/100ms
   unoptimized xpath     1.556k i/100ms
Calculating -------------------------------------
     optimized xpath     16.996k (± 1.9%) i/s -     85.085k in   5.008089s
   unoptimized xpath     16.038k (± 1.7%) i/s -     80.912k in   5.046467s

Comparison:
     optimized xpath:    16995.7 i/s
   unoptimized xpath:    16037.9 i/s - 1.06x  (± 0.00) slower

Weird.

flavorjones commented 3 months ago

OK, this has something to do with how expensive the node's context position is to calculate. If I replace //item with /root/item the benchmark shows optimization:

Warming up --------------------------------------
     optimized xpath     2.396k i/100ms
   unoptimized xpath     2.192k i/100ms
Calculating -------------------------------------
     optimized xpath     23.673k (± 2.0%) i/s -    119.800k in   5.062453s
   unoptimized xpath     21.616k (± 3.5%) i/s -    109.600k in   5.076759s

Comparison:
     optimized xpath:    23673.3 i/s
   unoptimized xpath:    21616.3 i/s - 1.10x  (± 0.00) slower

Shrug, I'm not sure it's worth making this part of at_css and at_xpath ... wrapping the query in (...)[1] doesn't seem like an automatic win.