angular / protractor-accessibility-plugin

Runs a set of accessibility audits
MIT License
268 stars 37 forks source link

Protractor suites and Accessibility-pluging #4

Open FJLopezGarcia opened 8 years ago

FJLopezGarcia commented 8 years ago

Is it possible to run protractor suites with the accessibility-pluging to get A11y vulnerabilities for each suite?

protractor protractor.conf.js --suite homepage

plugins: [
  {
    chromeA11YDevTools: {
      treatWarningsAsFailures: true
    },
    package: 'protractor-accessibility-plugin'
  }
],

I am receiving the following issue:

C:\MySolution_Git\>protractor config/protractor.config.js --suite dataPrivacy
[launcher] Process exited with error code 1
C:\Program Files\nodejs\node_modules\protractor\node_modules\q\q.js:126
                    throw e;
                    ^

Error: Cannot find module 'protractor-accessibility-plugin'
    at Function.Module._resolveFilename (module.js:325:15)
    at Function.Module._load (module.js:276:25)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at C:\Program Files\nodejs\node_modules\protractor\lib\plugins.js:36:19
    at Array.forEach (native)
    at new Plugins (C:\Program Files\nodejs\node_modules\protractor\lib\plugins.js:25:20)
    at Runner.run (C:\Program Files\nodejs\node_modules\protractor\lib\runner.js:250:17)
    at TaskRunner.run (C:\Program Files\nodejs\node_modules\protractor\lib\taskRunner.js:123:19)
    at createNextTaskRunner (C:\Program Files\nodejs\node_modules\protractor\lib\launcher.js:223:20)

But, If I gun a gulp task executing all my specs it runs BUT the Accessibility results are over the latest page:

gulp.task('test:e2e', function(cb) {
    gulp.src(['/test/javascript/e2e/*.spec.js']).pipe(protractor({
        configFile: __dirname + '/config/protractor.config.js',
        //args: ['--baseUrl', 'http://localhost:3383/'],
        debug: false
    })).on('error', function(e) {
        console.log(e);
    }).on('end', cb);
});
sjelin commented 8 years ago

Have you npm installed protractor-accessibility-plugin?

FJLopezGarcia commented 8 years ago

Yes I have it and the plugin is added to my protractor.conf as you can see above. I install the protractor-accessibility-plugin with npm install from my pakage.json

{
  "name": "UITestingFramework",
  "title": "UITestingFramework",
  "version": "0.0.3",
  "description": "UITesting Framework",
  "dependencies": {
    "gulp": "^3.5.6",
    "gulp-jshint": "^2.0.0",
    "gulp-jshint-xml-file-reporter": "^0.5.1",
    "gulp-protractor": "^2.1.0",
    "jasmine-spec-reporter": "^2.4.0",
    "jasmine-reporters": "^2.1.1",
    "jshint": "^2.9.1-rc3",
    "karma": "^0.13.11",
    "karma-coverage": "^0.5.2",
    "karma-chrome-launcher": "^0.2.2",
    "karma-firefox-launcher": "^0.1.7",
    "karma-ie-launcher": "^0.2.0",
    "karma-safari-launcher": "^0.1.1",
    "karma-htmlfile-reporter": "^0.2.2",
    "karma-jasmine": "^0.3.6",
    "karma-junit-reporter": "^0.3.7",
    "karma-webdriver-launcher": "~0.2.0",
    "phantomjs": "^1.9.19",
    "protractor": "^3.0.0",
    "protractor-accessibility-plugin": "^0.1.1",
    "protractor-jasmine2-html-reporter": "0.0.5"
  }
}
sjelin commented 8 years ago

This is very strange. The error message you're showing is saying that node cannot find the module, which normally means that it isn't installed.

FJLopezGarcia commented 8 years ago

yes.. that is only when I run Protractor pointing to am specifc Suite protractor protractor.conf.js --suite homepage

When I run "protractor protractor.conf.js" it works fine but the accessibility audit only runs for the latest page. Is there any way to run the Accessibility audit across all pages defined in the Protractor.conf ?

Thx

sjelin commented 8 years ago

Can you provide me with a minimal example, including the html/js files under test, spec file, and config file?

Gokul1987 commented 8 years ago

Hi Guys,

Is there a fix for the bug. I am currently facing the same issue, I got a test which will login and check the home page. When I run my protractor tests, I only get the accessibility audit for the home page(ie the last page ran by the test) and not for the login page.

But when I run just the login page, I get the accessibility issues for the login page.

Below I have attached just a very basic test of logging into application.

TODO-SPEC.JS

describe('Protractor demo app', function() {
     it('should add a todo', function() {
          browser.get('http://****.com/');
          expect(browser.getTitle()).toEqual('Loop QUOTA');
      });
      it('quota login page', function() {
           //steps for login page
           element(by.id("login_username")).sendKeys('****');
           element(by.id("login_password")).sendKeys('****');
           element(by.id('login_signIn')).click();
           //this step checks the home page
           var e = element(by.id('jnumber_jnumberrequest'));
           expect(e.isPresent()).toBeTruthy();
        });
});

conf.js

exports.config = {
  seleniumAddress: 'http://localhost:4444/wd/hub',

  // Capabilities to be passed to the webdriver instance.
  capabilities: {
    'browserName': 'firefox'
  },

  plugins: [{
   chromeA11YDevTools: {
     treatWarningsAsFailures: false
   },
   package: 'protractor-accessibility-plugin'
   }],

  framework: 'jasmine',

  specs: ['todo-spec.js'],

  jasminNodeOPts: {
    defaultTimeoutInterval: 30000
  }
};

Please help, as we need to decide a tool in a weeks time

Regards, Gokul

Gokul1987 commented 8 years ago

Hi guys,

Any updates on the issue.

sjelin commented 8 years ago

I'll do it monday

FJLopezGarcia commented 8 years ago

Hi @sjelin, Any update on this?

Thanks

sjelin commented 8 years ago

Sorry, I'll do it today this time ;P

FJLopezGarcia commented 8 years ago

Great, please keep us posted!! and thanks a lot!!

sjelin commented 8 years ago

@FJLopezGarcia, I cannot reproduce your error message. It also sounds like Gokul1987 managed to run the plugin on a specific suite without errors. Can you post a config/spec/webpage that reproduces your error?

Also, as so the problem that you only get a11y data on the last page, I'm working on it: https://github.com/angular/protractor/issues/2951

manoj9788 commented 8 years ago

I am getting the same error. Am able to see the protractor-accessibility-plugin folder inside node-modules folder but when I execute command Protractor conf.js - I am getting cannot find module error.

sjelin commented 8 years ago

Can you post a config/spec/webpage that reproduces your error?

danielnixon commented 8 years ago

@FJLopezGarcia It looks like you're using the global protractor. Have you tried ./node_modules/protractor/bin/protractor protractor.conf.js instead of protractor protractor.conf.js?

Gokul1987 commented 8 years ago

@sjelin hi mate..is there any updates on this bug...I have been told to write tests to monitor accessibility of our application..

jesselpalmer commented 8 years ago

I ran into this problem myself today. I had the protractor-accessibility-plugin installed locally, but I needed to install the package globally by running npm i -g protractor-accessibility-plugin to get protractor protractor.conf.js to work.

bmalone commented 8 years ago

I had this issue recently, as well, and jesselplamer's solution worked for me. After installing the protractor-accessibility-plugin module globally, running "protractor protractor.conf.js" successfully ran and gave me a Chrome A11Y audit for the last page. (NOTE: I haven't tried this for multiple specs at a time. I would expect it to only audit the last page in all tests, but that's unverified).

As for running on the wrong page, I hacked that to run on the right page: My test spec file does not have any code running the A11Y tools since it runs itself, but I did add a mostly empty describe() block at the end of my tests with a beforeAll() that handled logging the right user into the app and then navigating to the page I wanted. That way it ends in the exact state I want for my accessibility check.

Thanks @jesselpalmer !

sjelin commented 7 years ago

Hey, sorry guys, this bug just kinda fell of the map for me for some reason.

Yes, the problem is that the plugin is locally installed but Protractor is global, so Protractor can't find it. You can globally install the plugin, or you can switch from:

    package: 'plugin-name',

To:

    inline: require('plugin-name'),

I recommend the latter. I also wrote a Protractor issue about this: https://github.com/angular/protractor/issues/4090