Further work on test suite?

[danielvdende]

Hi guys,

It's been a while since we last did any work on this project, maybe we can continue our work? I'm up for doing some more work on it anyway. I think it would be great experience and I think it's great to stay in touch with people like Jasper. I drew up an initial outline of what I think still needs to be done:

code

• rebuild into new github repo
• proposed file structure:
• [PARENT] index.html (file that should allow users to select which type of documents to run through the test suite. Kind of UI so we can easily present the results) js/css files for index.html readme.md
  • [PAGES] folder that contains all pages that we have created so far + possible additions. Discuss with Jasper which pages are useful to add + add full jqm pages to run through the test.
  • [ANALYSIS] folder that contains all output of the test suite. Should be automated, so it should automatically write to a CSV or excel file.
  • [SUPPORTING CLASSES] Any supporting classes we need, such as the handler for writing excel from php
  • [JQUERY] Folder containing all jquery and jquery mobile files, freshly built from the github repo.

work that still needs to be done:

• refactor code (very broad concept, but includes redoing some code in order to create cleaner, more structured testing code)

blog post:

• intro
• background information
• what has changed in jqm 1.4 (see jasper's blog post)
• test plan
  • file structure
  • reference code
  • initial load time
  • click load time
• present results
  • statistical significance
• conclusions

[danielvdende]

No problem, I won't be able to complete the desktop tests until tomorrow either. @NathanMol Could you do the analysis when @vhogendoorn and me have finished running the extra test runs?

[NathanMol]

I'm willing to do the analysis.  Maybe someone else takes care for the test  where you asked me for?

Verzonden vanaf Samsung Mobile

-------- Oorspronkelijk bericht -------- Van: danielvdende notifications@github.com Datum: 18-09-2013 14:03 (GMT+01:00) Aan: danielvdende/jqmTestSuite jqmTestSuite@noreply.github.com Cc: NathanMol nathanmol@talpaedesign.nl Onderwerp: Re: [jqmTestSuite] Further work on test suite? (#1)

No problem, I won't be able to complete the desktop tests until tomorrow either. @NathanMol Could you do the analysis when @vhogendoorn and me have finished running the extra test runs?

— Reply to this email directly or view it on GitHub.

[vhogendoorn]

@danielvdende and I are doing the tests nathan.

[danielvdende]

@NathanMol We have finished running the extra test runs. Would you be able to quickly do the analysis? On first glance the results aren't positive at all. Doesn't seem to make any difference whether we use a desktop or use jquery 2.0.

What do you guys think we should do if the results remain like this?

[NathanMol]

Chrome desktop, it doesnt look good...

I will do the other analysis tonight

[danielvdende]

Ok, that indeed doesn't look fantastic. When @NathanMol has done the other analysis I'll send the results to Jasper, perhaps he can explain what has happened...

[NathanMol]

Complete overview off the tests. For Chrome desktop browser it would be useful to set the confidence interval to 90%. But the results of Chrome Mobile 2.0 are dramatic...

[danielvdende]

I mailed Jasper with the results, asking him if he knew of any reason why the performance has taken such a hit since early June. Hopefully he'll be able to give us some answers...

[danielvdende]

I re ran a few tests using jqm version 1.3.1 instead of 1.3.2 to see if that might be the reason for the decline in performance. I didn't do full analyses but based on quick examination I don't think this is the issue either. I uploaded the results to the dropbox if anyone is interested in taking a look. The rums were all on desktop in chrome.

[danielvdende]

Jasper indicated he's going to take a look at the results and the test suite as a whole on Friday ;). He said there shouldn't really be any difference between the old and new builds, but he also said that perhaps they had missed something. If, however, they did miss something, our test suite functioned well, as it highlighted a drop in performance fairly quickly.