Painfully slow

[baelter]

Can something be done to speed up the site? Can we start a donation round to sponsor better hosting? Or set up a mirror? Add caches?

[lsegal]

@baelter can you provide information on where you are seeing slow performance? Our metrics are currently showing an average response time of about 24ms and 622req/min which is about the best we've ever had on the site.

[taylorthurlow]

I'm also experiencing the same. I just loaded Sinatra docs here and it took 30+ seconds for the site to even paint any content. Other sites are unaffected. I'll try and replicate but it only happened once. It does seem to be a semi-recurring theme though.

I'll try and get a Firefox network analysis dump the next time I experience the problem.

[taylorthurlow]

Okay, here's a Firefox 75 HAR dump loading this page, and it took 40+ seconds to finish loading assets. Most HTML stuff looks like it finished early. My issue with the Sinatra docs I mentioned before was not similar, but it's something.

https://gist.github.com/taylorthurlow/84cc90a2040ea43fd89a296828cfac76

[baelter]

Sure, randomly choose net/ssh, took 17s for initial page to load. Then another 10s for css/js

https://gist.github.com/3a5f900a83bd481ffe902afd565aafe7

[lsegal]

I can see some occasional 5s slowdowns on very few requests temporally but those are usually only around for seconds at a time and refreshing resolves. I can't really reliably reduce slow speeds.

I do notice that from our metrics our site performs much slower at night (PST time), so there might be something there-- can everyone here let us know what times of day you are accessing the site (+ timezone) when it is slow for you? This may be related to some background processing tasks. TLDR: daytime performance during PST (GMT-8) seems pretty reasonable to me.

[taylorthurlow]

My tests were done around 4PM PST.

[baelter]

I usually use the site during CET office hours.

[janko]

I also often experience performance issues with rubydoc.info. It's always very slow for me when browsing the Devise and Thor docs.

[jage]

... can everyone here let us know what times of day you are accessing the site (+ timezone) when it is slow for you? This may be related to some background processing tasks. TLDR: daytime performance during PST (GMT-8) seems pretty reasonable to me.

Experiencing this as well. Collected some response time metrics the last 6 days, see screenshot. Timezone CEST (UTC+2).

Last 6 days, checked every 5 minutes. It expects at least one .object_link to be rendered before a response is "done". response times longer than 30s is marked as downtime.

[VxJasonxV]

It's incredibly sporadic, but I see this slowness as well. I'm looking into the best way to visualize it, probably just a screen recording with a timer or the browser dev tools or some such.

In addition to slowness there is another problem, occasional blank pages. I managed to capture it via BrowserStack:

https://www.browserstack.com/screenshots/0998d60b59d917118c8083a93f2f5720e2237c73

Image for posterity, I don't know how long that cache will exist:

MacOS Yosemite Safari is completely blank, no error. It won't happen again if you re-generate the screenshots. ex. https://www.browserstack.com/screenshots/f0d8d2d667d7e617610ac0492142f4433c2a9ec3

Sadly since this was automated I can't see what the code / error was via dev tools. But that represents the sporadic-ness I'm seeing, requests will just randomly be blank (or load slowly, or timeout, etc.) requiring a refresh.

I've been seeing it happen while I've been working on this project for the last ~2 hours (06:00 - 08:00 UTC). I've been using a combination of Chrome and Safari tonight, I honestly could not tell you which one(s) I've experienced the problem in, but it's probably both, but definitely Chrome because that's what I have over ⬅️ right now.

Again, random/sporadic, and likely not client dependent!

[Zeragamba]

Loading the RubyDoc.info homepage from Kitchener-Waterloo, Canada at about 17:00 EST took 43 seconds to load all assets.

[lsegal]

We made some changes yesterday that probably won't fully fix performance issues but should smooth out some of the rough edges-- curious to see if anyone here is still noting any issues?

[baelter]

Initial feeling is promising

[lsegal]

This can be closed since it's either stale or has been addressed. This isn't to say there are no performance issues, but we addressed some of the major ones. Please open new issues if you notice problems again, thanks!

[baelter]

Still pretty poor imo :/

[lsegal]

@baelter What site is this? The analytics we're reporting show a pretty different average and much more inline with measured behavior. A "low" of ~2.5 sec doesn't match with real-world measurements of pages fully loading in ~600ms:

Is this metric measuring full page load? Is it including external assets? Is it hitting a specific URL? Could there be some inherent base latency in all the requests for this site?

[baelter]

This is an uptime.com transaction check from Sweden that loads https://rubydoc.info/github/net-ssh/net-ssh and waits for .object_link elements to exist. Matches my experience (also from Sweden) pretty well. 2s is ok I suppose but quite often it spikes over 10s. net-ssh is one of the faster once. This one takes ~30s https://rubydoc.info/gems/aws-sdk-ec2

[lsegal]

Measuring individual libraries on generate time is going to depend very much on the size of the libraries themselves-- that's not a limitation of rubydoc's server speed but a limitation of generate time in general. When rubydoc is regenerating content for large codebases like the AWS EC2 SDK, it's very likely that it can take upwards of 20s. Because content is generated on demand, individual pages have to still be rendered and cached after initial generation, which can still take upwards of a few seconds depending on the size of the library (de-serializing the YARD db can be the limiting factor here, esp for extremely large libraries like EC2). There's not much we can specifically do about that one at all times.

In order to mitigate, we cache heavily but will always have to release cache and regenerate when the gems are updated (and sometimes we release cache to regenerate and keep data fresh in case of corruption or bad generation). This should keep performance fast but there are cases where time will spike.

For reference right now the EC2 SDK is generating individual pages in ~1.5s and cached pages are loading in ~500ms. The Net::SSH lib is doing a lot better, loading even uncached pages in ~600ms and cached ones in about 300ms.

We expect to see spikes in performance when we do cache invalidation or when we refresh a lot of gems, plus occasionally servers go for rolling restarts behind the scenes and if you catch the site at a particularly bad time it can spike, but I still don't see the min-fetch time at 2s unless we're considering heavy latency on the fetching side.

In other words, there are of course still improvements we can make, but most of the day we are seeing fairly normal performance for the site, and our "average" load times seem to be pretty consistent; certainly much more consistent than when this issue was filed.

[baelter]

Thanks for all the detail! Didn't consider that the docs were generated on demand, that explains the variance.

Keep up the good work 👍

[baelter]

Could you cache for longer? Seems like the site is generating docs way more often than new versions are released? Looking at our load times looks like it's re-generated almost every request?