Scalability Audit - Githubissues

cdelahousse commented 11 years ago

Imported issue by spratt from 2012-07-06T16:01:18Z Originally opened as https://github.com/SteamedPears/CodeReviewClientMaster/issues/34

We really need to sit down and try to hammer the site, to make sure it can handle a slashdot.

spratt commented 11 years ago

I think @cdelahousse was mentioning Apache bench https://httpd.apache.org/docs/2.2/programs/ab.html running on any computers we can get ahold of, or some ec2 instances.

psimonyi commented 11 years ago

I have access to some servers in Waterloo we could use.

spratt commented 11 years ago

@cdelahousse @psimonyi @bheesham One or more of us should do this for our next (and final) sprint. I want to get @asomayaji to give us feedback on how to scale before the end of the course.

spratt commented 11 years ago

Part of this should address #33 and #45 and #64

spratt commented 11 years ago

http://stackoverflow.com/questions/4710420/scaling-node-js

spratt commented 11 years ago

@cdelahousse You're lead on this.

I have a conversion to redis back-end blocked waiting for another pull request to go through. Once that's in, we should arrange to spend a night this week hammering the app running on my laptop (since we don't have a production deployment ready yet).

cdelahousse commented 11 years ago

Spending a night together hammering a machine sounds really nice.

I wanna know what you mean by scalability audit. I mean, there s a lot we could do. You spoke of minification in another issue and related it to the audit. I'm confused as to how the relates to auditing.

Should this issue be "Make scale and then test"? On Mar 25, 2013 12:49 AM, "Simon David Pratt" notifications@github.com wrote:

@cdelahousse https://github.com/cdelahousse You're lead on this.

I have a conversion to redis back-end blocked waiting for another pull request to go through. Once that's in, we should arrange to spend a night this week hammering the app running on my laptop (since we don't have a production deployment ready yet).

— Reply to this email directly or view it on GitHubhttps://github.com/CarletonU-COMP2406-W2013/Steamed-Pears/issues/31#issuecomment-15377765 .

spratt commented 11 years ago

On 03/26/2013 08:52 AM, Christian Delahousse wrote:

Spending a night together hammering a machine sounds really nice.

Heh.

I wanna know what you mean by scalability audit. I mean, there s a lot we could do. You spoke of minification in another issue and related it to the audit. I'm confused as to how the relates to auditing.

I mean we should thoroughly examine (audit) the bottlenecks of scaling the app and, implicitly, to fix as many as we can. w.r.t. minification

it's something on which we can improve, it should be a goal to figure out the best way to do it in production and do it.

psimonyi commented 11 years ago

You should check how the redis snapshot configuration performs. /etc/redis/conf:108

spratt commented 11 years ago

Using sqlite

siege -b -c 200 -t60S http://192.168.1.103:8080/do/code?id=1

Transactions: 10378 hits Availability: 100.00 % Elapsed time: 59.53 secs Data transferred: 1.39 MB Response time: 1.13 secs Transaction rate: 174.33 trans/sec Throughput: 0.02 MB/sec Concurrency: 197.04 Successful transactions: 10378 Failed transactions: 0 Longest transaction: 9.23 Shortest transaction: 0.02

Using redis

siege -b -c 200 -t60S http://192.168.1.103:8080/do/codeById?id=04a34cfa-6cc6-438d-9b1d-0cae1dfeeb5a

Transactions: 12925 hits Availability: 100.00 % Elapsed time: 59.86 secs Data transferred: 1.01 MB Response time: 0.92 secs Transaction rate: 215.92 trans/sec Throughput: 0.02 MB/sec Concurrency: 198.35 Successful transactions: 12925 Failed transactions: 0 Longest transaction: 6.75 Shortest transaction: 0.05

spratt commented 11 years ago

Sources of Errors

The above tests were with very similar but not identical data.
The second test (redis) was immediately following a system reboot and while running a GUI (lightdm)
A significant amount of the response string was taken up by pieces of data that sequelize was adding to the models (last modified time and created time)

Discussion of Results

I think the above captures that the redis implementation is faster, but much of this gain might simply be since we're serving less data, not because of the change in persistent store.

Ideas for Future Testing

Test with identical data
Test with identical response size (including headers, which may be different)
Test consistently following a system reboot and with no GUI
Compare the commentCount response time in redis to the comments response time in sqlite, with identical number of comments. These calls are harder on the database, so they will probably be a bigger bottleneck.
Test creation of new code/comment response time

spratt commented 11 years ago

I think @bheesham had exactly the right idea when he suggested using headless webkit, for testing the page load time performance. I think it's important to test page load time because it has a big impact on user experience. We should do a before and after the minification.

@bheesham: which headless webkit thing were you suggesting? PhantomJS (http://phantomjs.org/)? Is there an automated testing tool for page load you would recommend? I'm looking at YSlow (http://yslow.org/) and confess.js (https://github.com/jamesgpearce/confess)

spratt commented 11 years ago

@cdelahousse I think we've ironed out the major bug with your dev environment. Do you think you can get some data on page load times before/after minification? I think we could bring our data to @asomayaji with the things we've considered and he can (hopefully) suggest where to go from there.

spratt commented 11 years ago

The nice thing about page load times is that they aren't about stressing your system. We should just load the page > 30 times and take the average of the page load from initial request to final response (or ideally the end of rendering if possible?).

spratt commented 11 years ago

We should figure out how to cache requests on the back-end. There's not a lot of data, but some of the requests could take a lot of time. For instance, the commentCount call takes time linear w.r.t. the number of lines on which there are comments. This might get slow. We should test how slow it is versus how many lines have comments.

cdelahousse commented 11 years ago

Page load testing: I'm using Google Chrome with Chrome Dev Tools. I'll clear my cache on every reload. I'm running Code Review locally.

These are the results for the project at commit da8e84c401486eaac706bfc2b8ae98a6ef374ef5

I'm using 5 paragraphs of lorem ipsum. (http://www.lipsum.com)

Unminified

The following is the project before the production script has been run.

Chrome tools audit:

    Network Utilization

    Combine external CSS (4)

        There are multiple resources served from same domain. Consider combining them into as few files as possible.
            4 CSS resources served from localhost.

        Enable gzip compression (1)
        Compressing the following resources with gzip could reduce their transfer size by about two thirds (~1.6 KB):
            index.html could save ~1.6 KB

        Leverage browser caching (6)
            The following resources are explicitly non-cacheable. Consider making them cacheable if possible:
            index.html
            codemirror.css
            view.css
            introjs.min.css
            bootstrap.min.css
            require.js

        Minimize cookie size
            The average cookie size for all requests on this page is 44 B

        Web Page Performance

        Remove unused CSS rules (1448)
            89.2 KB (79%) of CSS is not used by the current page.
                codemirror.css: 5.1 KB (100%) is not used by the current page.
                view.css: 1.1 KB (55%) is not used by the current page.
                introjs.min.css: 3.6 KB (91%) is not used by the current page.
                bootstrap.min.css: 79.5 KB (78%) is not used by the current page.
                Inline block #1: 0 B (0%) is not used by the current page.

        Use normal CSS property names instead of vendor-prefixed ones (1)
            introjs.min.css
                .introjs-button
                    "-webkit-background-clip" is used, but "background-clip" is supported.

For loading times, I'm using the Chrome Timeline and an average of a few reloads. I've disabled my cache.

24 requests ❘ 575 KB transferred ❘ 1.50 s (onload: 885 ms, DOMContentLoaded: 885 ms) 24 requests ❘ 575 KB transferred ❘ 1.79 s (onload: 1.11 s, DOMContentLoaded: 1.11 s) 24 requests ❘ 575 KB transferred ❘ 1.35 s (onload: 667 ms, DOMContentLoaded: 667 ms) 24 requests ❘ 575 KB transferred ❘ 1.51 s (onload: 851 ms, DOMContentLoaded: 849 ms) 24 requests ❘ 575 KB transferred ❘ 1.42 s (onload: 658 ms, DOMContentLoaded: 657 ms) 24 requests ❘ 575 KB transferred ❘ 1.25 s (onload: 581 ms, DOMContentLoaded: 577 ms)

8.82/6=1.47 seconds.

BTW, I got to use awk for that. I'm happy.

Minified version coming up!

cdelahousse commented 11 years ago

Minified version

Same setup for this test, except I've run the production script and replaced the contents of src/client with the production output.

Dev Tools audit output:

Network Utilization

Combine external CSS (4)

    There are multiple resources served from same domain. Consider combining them into as few files as possible.
        4 CSS resources served from localhost.

Enable gzip compression (6)
    Compressing the following resources with gzip could reduce their transfer size by about two thirds (~87.2 KB):
        index.html could save ~1.6 KB
        codemirror.css could save ~3.1 KB
        view.css could save ~1.3 KB
        introjs.min.css could save ~2.7 KB
        bootstrap.min.css could save ~69.0 KB
        require.js could save ~9.5 KB

Leverage browser caching (6)
    The following resources are explicitly non-cacheable. Consider making them cacheable if possible:
        index.html
        codemirror.css
        view.css
        introjs.min.css
        bootstrap.min.css
        require.js

Minimize cookie size
    The average cookie size for all requests on this page is 44 B

Web Page Performance

Remove unused CSS rules (1448)
    88.7 KB (79%) of CSS is not used by the current page.
        codemirror.css: 4.5 KB (100%) is not used by the current page.
        view.css: 1.1 KB (55%) is not used by the current page.
        introjs.min.css: 3.6 KB (91%) is not used by the current page.
        bootstrap.min.css: 79.5 KB (78%) is not used by the current page.
        Inline block #1: 0 B (0%) is not used by the current page.

Use normal CSS property names instead of vendor-prefixed ones (1)
    introjs.min.css
        .introjs-button
            "-webkit-background-clip" is used, but "background-clip" is supported.

From the timeline:

9 requests ❘ 421 KB transferred ❘ 1.03 s (onload: 563 ms, DOMContentLoaded: 563 ms) 9 requests ❘ 421 KB transferred ❘ 1.20 s (onload: 684 ms, DOMContentLoaded: 683 ms) 9 requests ❘ 421 KB transferred ❘ 1.13 s (onload: 595 ms, DOMContentLoaded: 594 ms) 9 requests ❘ 421 KB transferred ❘ 1.03 s (onload: 585 ms, DOMContentLoaded: 585 ms) 9 requests ❘ 421 KB transferred ❘ 1.07 s (onload: 561 ms, DOMContentLoaded: 560 ms) 9 requests ❘ 421 KB transferred ❘ 1.22 s (onload: 682 ms, DOMContentLoaded: 680 ms)

6.68/6=1.11s per page load!

We've obviously improved by a lot!

cdelahousse commented 11 years ago

A few things we could do.

Server side:

enable far-expiring caching with versioning on all JS and CSS
enable gzip compression

Clientside:

I think our r.js build profile needs a bit of work. Have you see what it does to main.js? There's like 3 pages of licenses.
I'm wondering if there's a bootstrap optimizer. We're only using a small part of it. We could probably slice out what we don't need.

cdelahousse commented 11 years ago

A few more things:

I first read these rules a few years ago. Found them cool: http://developer.yahoo.com/performance/rules.html

The same thing but from google: https://developers.google.com/speed/docs/best-practices/rules_intro

@bheesham was speaking of building a phantomJS test bed. YSlow has basically done the same thing. http://yslow.org/node-server/ http://yslow.org/command-line-har/ http://yslow.org/phantomjs/

spratt commented 11 years ago

We should merge and optimize the CSS files. That will get rid of 3 requests.

psimonyi commented 11 years ago

gzip should make the repeated licences pretty much negligible. nginx should be able to do the caching stuff.

spratt commented 11 years ago

We don't just have repeated licenses. The massive comment headers we have at the top of each client js file are copied into the minified file.

SteamedPears / Code-Review

Scalability Audit #31

Sources of Errors

Discussion of Results

Ideas for Future Testing

Unminified

Minified version