search

optimizely / ruby-sdk

Ruby SDK for Optimizely Feature Experimentation and Optimizely Full Stack (legacy)
https://docs.developers.optimizely.com/experimentation/v4.0.0-full-stack/docs/ruby-sdk
Apache License 2.0
24 stars 29 forks source link

Slowness after upgrading to 3.9 #295

Closed integralhero closed 2 years ago

integralhero commented 2 years ago

Hey there, we recently bumped to 3.9 and noticed latency spike (~double) along a critical code-path. We have json schema validation disabled here for performance reasons.

As a comparison, here's a flamegraph I made where our code uses

Release version 3.8 of the ruby-sdk gem: Screen Shot 2022-01-24 at 10 50 32 AM

vs. 3.9. Screen Shot 2022-01-24 at 10 50 43 AM

I see a lot of cycles spent in Optimizely::StaticProfileManager, possibly tied to this change

I was wondering if anyone's noticed something similar/ could give me more context on the change here and what we might be able to do to address the performance issues

msohailhussain commented 2 years ago

Can you please share flamegraph SVGs of both releases and it will be helpful if you send diff as well using difffolded.pl

integralhero commented 2 years ago

Here's a zip containing flamegraphs for 3.8, 3.91, and 3.10

flamegraph.zip

Danny-Driscoll commented 2 years ago

@integralhero Re-opening after the auto-close from the PR merge. We just merged a change that we believe will improve performance on the initialization process for the SDK by deferring some data processing until it is explicitly accessed vs. running it on initialization. The data processing wasn't entirely new in 3.8 but we did expand the scope of data being processed in that version, so we suspect it would help address a change in that version. It's on Master now if you are able to validate off that whether you see an improvement in your test setup.

integralhero commented 2 years ago

Thanks so much for following up @Danny-Driscoll! Just tried it out and it looks fine. Cheers!

integralhero commented 2 years ago

Could we incorporate this in the next minor version bump? (not sure how y'all do releases)

Danny-Driscoll commented 2 years ago

Yep, this will be released shortly in a new minor version. We just wanted to get some validation that it did indeed improve your observed performance before pushing it out.

zashraf1985 commented 2 years ago

@integralhero We just released 3.10.1 which contains this change. https://github.com/optimizely/ruby-sdk/releases/tag/3.10.1 https://rubygems.org/gems/optimizely-sdk/versions/3.10.1

integralhero commented 2 years ago

Thanks! Closing this issue