Karma

[rogerxu]

Karma - Spectacular Test Runner for Javascript

• [x] Mocha
• [x] QUnit
• [x] OpenUI5
• [x] PhantomJS
• [x] JUnit reporter
• [x] Coverage
• [x] Proxies

[rogerxu]

Configuration

Karma - Configuration File

Generate configuration file

$ karma init karma.conf.js

Start Karma

$ karma start karma.conf.js

Config

config.set(newObject); // call _.mergeWith(oldObject, newObject);

Properties are exposed by config object.

config.basePath
config.files

Files

Karma - Files

• basePath - The root path location that will be used to resolve all relative paths defined in files and exclude.
• files - List of files/patterns to load in the browser.
  • pattern - The relative path from basePath or absolute path
  • watched: true
  • included: true - Included in the browser using <script> tag.
  • served: true
  • nocache: false
• exclude - List of files/patterns to exclude from loaded files.

If pattern is outside the basePath, it will be in /absolute path, otherwise in /base in the server.

Browsers

• PhantomJS - karma-phantomjs-launcher
• Chrome - karma-chrome-launcher
• Firefox

{
  browsers: ['Chrome_without_security'],

  customLaunchers: {
    Chrome_without_security: {
      base: 'Chrome',
      flags: ['--disable-web-security'],
      displayName: 'Chrome without security'
    }
  }
 }

Frameworks

• mocha
• qunit - karma-qunit
• openui5 - karma-openui5

Preprocessors

• coverage - karma-coverage

preprocessors: {
 '**/*.js': ['coverage']
}

Reporters

• progress
• spec - karma-spec-reporter
• junit - karma-junit-reporter
• coverage - karma-coverage

reporters: ['spec', 'coverage']
reporters: ['progress', 'junit', 'coverage']

Proxies

proxies: {
  '/img': '/base/test/images',
  '/resources': {
    target: 'http://cdn.example.com',
    changeOrigin: true
  }
}

Proxy can only apply on the local server path or absolute HTTP resource URL.

proxies: {
  '/resources': {
    target: '/absolute' + path.resolve('target/sapui5/resources'),
    changeOrigin: true
  }
}

[rogerxu]

How It Works

Karma - How It Works

karma/thesis.pdf at master · karma-runner/karma

Karma is essentially a tool which spawns a web server that executes source code against test code for each of the browsers connected. The results of each test against each browser are examined and displayed via the command line to the developer such that they can see which browsers and tests passed or failed.

Each browser loads the source files inside an IFrame, executes the tests and reports the results back to the server.

[rogerxu]

javascript - Karma vs testing framework Jasmine, Mocha, QUnit - Stack Overflow

Karma is a browser test runner.

The idea is that browsers do not have natively a concept of loading tests files, running them, and reporting results. What karma does is (roughly) :

• starting a small web server to serve "client-side" javascript files to be tested (1)
• also serve the "client-side" javascript files with the tests (or Specs, as they're often called) (2)
• serve a custom web page that will run javascript code for the tests (3)
• start a browser to load this page (4)
• report the results of the test to the server (5)
• karma can then again report the results to text files, the console, anything your CI server likes, etc...

Looking at each part :

1. Those files will be your actual js files ; you will tell karma how to load them. If you use requirejs, there is a karma plugin, and some config is needed.
2. Those tests can be written in a variety of Javascript testing framework (Jasmine, QUnit, Mocha) ; this is JS code that is run in the browser.
3. The custom web page will be a bit different for each testing framework ; this is why karma has plugins for different frameworks.
4. Karma can launch the page in many browsers (FF, Chrome, or headless browsers like PhantomJs.)
5. Reporting to karma is, again, framework-dependant, and dealt with karma plugins.

So to answer your questions :

• in Java, most people use JUnit which is both a framework to write tests and run them, but does not have the problem of differentiating the environment in which tests are run and the one in which test reports are aggregated ; karma would be the missing piece between a JUnit Suite and a JUnit TestRunner
• Yes, you can do everything that karma does "by hand" - pick one framework (jasmine, qunit, mocha) and follow instructions. The advantage of karma is that it provide a solution out-of-the-box, if you're in a standard setup.
• Karma can be used for both unit test (with jasmine / qunit / whatever) and integration tests (which will use another API, like webdriver, to drive the browser)

[rogerxu]

QUnit

karma-qunit

karma.conf.js

module.exports = function(config) {
  var showUI = true;

  config.set({
    // frameworks to use
    frameworks: ['qunit'],

    // How the client should execute the test, like run in an iframe, capture the console and so on.
    client: {
      captureConsole: true,
      clearContext: !showUI,
      useIframe: false,
      qunit: {
        showUI: showUI,
        testTimeout: 5000,
        autostart: false,
      },
    },
  });
};

Note: showUI: true needs the clearContext: false option to display correctly in non-debug mode.

[rogerxu]

Mocha

module.exports = function(config) {
  var showUI = true;

  config.set({
    // frameworks to use
    frameworks: ['mocha'],

    files: [
      'test/**/*.spec.js'
    ],

    reporters: ['spec', 'coverage'],

    coverageReporter: {
      dir : '../../coverage',
      subdir: function(browser) {
        // If only PhantomJS is used, remove the folder with the browser name
        return '';
      },
      reporters: [
        {
          type: 'text-summary'
        },
        {
          type: 'text',
          file: 'coverage.txt'
        },
        {
          type: 'lcov',
          subdir: 'lcov'
        }
      ]
    },
  });
};

[rogerxu]

OpenUI5

Headless OPA5 testing with Karma and PhantomJS | SAP Blogs

karma.conf.js

module.exports = function(config) {
  var showUI = true;

  config.set({
    // base path that will be used to resolve all patterns (eg. files, exclude)
    basePath: 'webapp',

    // frameworks to use
    frameworks: ['openui5', 'qunit'],

    // list of files / patterns to load in the browser
    files: [
      {
        pattern: '**',
        included: false
      }
    ],

    // list of files to exclude
    exclude: [],

    // preprocess matching files before serving them to the browser
    preprocessors: {},

    // In case an absolute URL is used at some point of the code, a proxy configuration is required.
    proxies: {
      '/resources': {
        target: '/base/resources',
        changeOrigin: true
      },
      '/test-resources': {
        target: '/base/test-resources',
        changeOrigin: true
      },
      '/base/resources': {
        target: `${custom_ui5}/resources`,
        changeOrigin: true
      },
      '/base/test-resources': {
        target: `${custom_ui5}/test-resources`,
        changeOrigin: true
      }
    },

    // test results reporter to use
    // possible values: 'dots', 'progress'
    reporters: ['progress'],

    // start these browsers
    // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
    browsers: ['Chrome'],

    openui5: {
      path: 'https://sapui5.hana.ondemand.com/resources/sap-ui-core.js', // only support absolute path for remote URL, relative path will be resolved as local path
      useMockServer: false
    },

    // How the client should execute the test, like run in an iframe, capture the console and so on.
    client: {
      captureConsole: true,
      clearContext: !showUI,
      useIframe: false,
      qunit: {
        showUI: showUI,
        testTimeout: 5000,
        // autostart: false,
      },
      openui5: {
        config: {
          theme: 'sap_belize_plus',
          libs: 'sap.m, sap.ui.layout, sap.ushell', // important to load more libs to avoid cross site origin issues
          compatVersion: 'edge',
          frameOptions: 'deny',
          preload: 'async',
          animation: false,
          debug: false,
          resourceRoots: {
            'demo/app': './base'
          }
        },
        tests: [
          'demo/app/test/unit/AllTests',
          'demo/app/test/integration/AllJourneys',
        ]
      }
    },

    // enable / disable colors in the output (reporters and logs)
    colors: true,

    // level of logging
    // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
    logLevel: config.LOG_DEBUG,

    // enable / disable watching file and executing tests whenever any file changes
    autoWatch: true,

    // Continuous Integration mode
    // if true, Karma captures browsers, runs the tests and exits
    singleRun: false,

    // Concurrency level
    // how many browser should be started simultaneous
    concurrency: Infinity,

    browserConsoleLogOptions: {
      level: 'warn'
    },

    // How long will Karma wait for a message from a browser before disconnecting from it (in ms).
    browserNoActivityTimeout: 30000
  });
};

webapp/test/unit/AllTests.js

// jQuery.sap.require("sap.ui.quit.qunit-coverage");
// jQuery.sap.require("sap.ui.thirdparty.sinon");

sap.ui.define([
  "./constant/tests"
], function() {
  "use strict";
});

[rogerxu]

JUnit Report

karma-junit-reporter

karma.conf.js

module.exports = function(config) {
  config.set({
    // test results reporter to use
    // possible values: 'dots', 'progress'
    reporters: ['progress', 'junit'],

    // the default configuration
    junitReporter: {
      outputDir: '../target/surefire-reports', // results will be saved as $outputDir/$browserName.xml
      outputFile: 'TEST-QUnitTest.xml', // if included, results will be saved as $outputDir/$browserName/$outputFile
      suite: '', // suite will become the package name attribute in xml testsuite element
      useBrowserName: false, // add browser name to report and classes names
      nameFormatter: undefined, // function (browser, result) to customize the name attribute in xml testcase element
      classNameFormatter: undefined, // function (browser, result) to customize the classname attribute in xml testcase element
      properties: {} // key value pair of properties to add to the <properties> section of the report
    }
  });
};

[rogerxu]

Coverage

karma-coverage

karma.conf.js

module.exports = function(config) {
  config.set({
    // preprocess matching files before serving them to the browser
    preprocessors: {
      'webapp/*.js': ['coverage'],
      'webapp/!(localService|test)/**/*.js': ['coverage'],
    },

    // test results reporter to use
    // possible values: 'dots', 'progress'
    reporters: ['progress', 'coverage'],

    coverageReporter: {
      dir: 'coverage',
      subdir: function(browser) {
        // If only PhantomJS is used, remove the folder with the browser name
        return '';
      },
      includeAllSources: true,
      reporters: [
        {
          type: 'html',
          subdir: 'html'
        },
        {
          type: 'text'
        },
        {
          type: 'text-summary'
        },

        {
          type: 'cobertura',
          subdir: '.',
        },
        {
          type: 'lcovonly',
          subdir: '.'
        }
      ],
      check: {
        global: {
          statements: 80,
          branches: 0,
          functions: 0,
          lines: 0,
        },
        each: {
          statements: 80,
          branches: 0,
          functions: 0,
          lines: 0,
        }
      }
    },

    // enable / disable watching file and executing tests whenever any file changes
    autoWatch: false,

    // Continuous Integration mode
    // if true, Karma captures browsers, runs the tests and exits
    singleRun: true,
  });
};

[rogerxu]

PhantomJS

karma-phantomjs-launcher

karma.conf.js

module.exports = function(config) {
  config.set({
    // start these browsers
    // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
    browsers: ['PhantomJS_custom'],

    customLaunchers: {
      'PhantomJS_custom': {
        base: 'PhantomJS',
        displayName: 'Custom PhantomJS',
        options: {
          // windowName: 'my-window',
          settings: {
            webSecurityEnabled: true
          },
        },
        flags: [
          // '--proxy=proxy:8080',
          // '--proxy-type=http'
        ],
        debug: false
      }
    },

    phantomjsLauncher: {
      // Have phantomjs exit if a ResourceError is encountered (useful if karma exits without killing phantom)
      exitOnResourceError: true
    },

    // Continuous Integration mode
    // if true, Karma captures browsers, runs the tests and exits
    singleRun: true,

    // How long will Karma wait for a message from a browser before disconnecting from it (in ms).
    browserNoActivityTimeout: 30000,

    browserConsoleLogOptions: {
      level: 'warn'
    },

    logLevel: 'INFO',
  });
};

Options

The options attribute allows you to initialize properties on the phantomjs page object, so

options: {
  windowName: 'my-window',
  settings: {
    webSecurityEnabled: false
  },
}

is equivalent to:

var webPage = require('webpage');
var page = webPage.create();

page.windowName = 'my-window';
page.settings.webSecurityEnabled = false;

[rogerxu]

ES6 Module

babel-plugin-transform-es2015-modules-umd

[rogerxu]

OPA5

Headless OPA5 testing with Karma and PhantomJS | SAP Blogs