danielpclark / faster_path

Faster Pathname handling for Ruby written in Rust
MIT License
780 stars 29 forks source link

Faster extname #153

Closed glebm closed 6 years ago

glebm commented 6 years ago

Based on top of #151

danielpclark commented 6 years ago

FYI I get performance improvement from between 30% to 60% on my version, so I marked the README as 40%. You can see from my Travis CI records it recorded a 60.4% performance boost. The proof.

The performance difference swings pretty widely.

Looking at yours I see 70.8%. That's pretty impressive.

glebm commented 6 years ago

I wonder if we can somehow make the benchmark more consistent? Increase the number of iterations perhaps?

danielpclark commented 6 years ago

That is a possibility. One problem is when we start breaking raw metal speeds slight differences look a lot bigger unless we make the benchmarks run for a very long time. Also we've noticed differences based on system CPU cache which may vary between whats available in the system at that moment.

We might need a flag feature on the pinch bench test for --long-run to sift out volatility. That would be great for Travis CI stats but then we have to wait that much longer for our builds to pass.

IDEA: We can make the --long-run flag only on the master branch. This won't delay any PRs that way and we still get the stats.

glebm commented 6 years ago

Looking into it a bit more, perhaps volatility is not the issue. Here are some numbers from a Travis Linux Ruby 2.5.0 run:

64-bit ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
64-bit rustc 1.24.1 (d3ae9a9e0 2018-02-27)
architecture: x86_64-unknown-linux-gnu
--------------------------------------------------------------------------------
Performance change for allocate, instead of new, is 84.2%
Performance change for absolute? is 95.8%
Performance change for add_trailing_separator is 68.7%
Performance change for basename is 49.4%
Performance change for children is 39.2%
Performance change for children_compat is -19.6%
Performance change for chop_basename is 76.6%
Performance change for cleanpath_aggressive is 78.2%
Performance change for cleanpath_conservative is 74.1%
Performance change for del_trailing_separator is 87.7%
Performance change for directory? is 16.8%
Performance change for dirname is 28.9%
Performance change for entries is 36.1%
Performance change for entries_compat is -195.3%
Performance change for extname is 66.2%
Performance change for has_trailing_separator? is 86.3%
Performance change for join is 67.9%
Performance change for plus is 86.6%
Performance change for relative? is 87.2%
Performance change for relative_path_from is 74.0%

These are very different relative to each other from what I get locally. If Travis does virtualization with CPU sharing, perhaps that would explain the variability.

64-bit ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
64-bit rustc 1.24.1 (d3ae9a9e0 2018-02-27)
architecture: x86_64-unknown-linux-gnu
--------------------------------------------------------------------------------
Performance change for allocate, instead of new, is 82.3%
Performance change for absolute? is 93.4%
Performance change for add_trailing_separator is 40.6%
Performance change for basename is 12.1%
Performance change for children is 39.7%
Performance change for children_compat is -35.3%
Performance change for chop_basename is 73.1%
Performance change for cleanpath_aggressive is 72.7%
Performance change for cleanpath_conservative is 73.6%
Performance change for del_trailing_separator is 82.6%
Performance change for directory? is 10.0%
Performance change for dirname is 43.4%
Performance change for entries is 24.0%
Performance change for entries_compat is -63.2%
Performance change for extname is 64.1%
Performance change for has_trailing_separator? is 74.6%
Performance change for join is 63.8%
Performance change for plus is 78.4%
Performance change for relative? is 82.2%
Performance change for relative_path_from is 69.0%

Compare basename and dirname to the Travis numbers. My local runs are consistent within ~5% usually.

danielpclark commented 6 years ago

Yeah… the stats on Travis do seem much more volatile. I wonder which language is affected more by their environment/system — Is Ruby slower because of it, or Rust faster?

glebm commented 6 years ago

Seems like these variations on Travis are caused by preemption from other VM instances, so it's random.

danielpclark commented 6 years ago

Ah yes. The processor Spectre and Meltdown issues.

danielpclark commented 6 years ago

By the way the only reason I rewrote your version of extname from earlier is when I was adding in official tests only the extname method failed the test suite. I'm glad you were able to make this much more concise version of it now. :smile:

coveralls commented 6 years ago

Coverage Status

Coverage remained the same at 93.023% when pulling b769c29215408982654ac85659aaf768d411b7ce on glebm:faster-extname into 0e2f44e54e3e82af6de5256c4b07ac60a75c5d6c on danielpclark:master.