search

jemishgopani / wdio-json-reporter

A WebdriverIO plugin. Report results in json format. http://webdriver.io
13 stars 32 forks source link

Make reporter compatible with WebdriverIO v5 #22

Closed christian-bromann closed 5 years ago

christian-bromann commented 5 years ago

Hola 👋

WebdriverIO released v5 around Christmas, see also this blogpost. To make this reporter usable with the new version there are some small adjustments necessary. There are some minor changes that need to be made. You can find a general overview here. Most important ones to point out are:

Feel free to drop by the webdriverio/v5 support channel in case you need support.

Thanks so much for contributing to the WebdriverIO community. We really appreciate it 🙏 !

fijijavis commented 5 years ago

@christian-bromann I started working on this today. I'm going to include this as a package in the v5 mono-repo. Do you know where I can find the structure of the runnner object?

christian-bromann commented 5 years ago

I'm going to include this as a package in the v5 mono-repo

Feel free but be aware that you need to help us maintain this then. There is technically no need to bring it under the @wdio umbrella. Since this reporter is fairly lightweight I don't mind.

pmerwin commented 5 years ago

@fijijavis anything I can do to help you with this?

asheeshsingh2014 commented 5 years ago

@fijijavis I`m looking forward to this reporter in v5 too. Please feel free to let me know if you think I can be of any help in this regard.

fijijavis commented 5 years ago

@pmerwin @asheeshsingh2014 Do either of you have experience with the v5 @wdio/reporter package?

fijijavis commented 5 years ago

@pmerwin @asheeshsingh2014 I am going to try to tackle this today

pmerwin commented 5 years ago

Great! @fijijavis sorry I am not familiar @wdio/reporter, but I would love to help review and test, and anything else you may need to get this ported.

fijijavis commented 5 years ago

@christian-bromann For now I'm going to keep this as a it's own project. When I publish to NPM is there any problem publishing as @wdio/json-reporter instead of wdio-json-reporter?

I'm thinking the name change might help alleviate any problems with folks who are still on WDIO v4

christian-bromann commented 5 years ago

When I publish to NPM is there any problem publishing as @wdio/json-reporter instead of wdio-json-reporter?

@fijijavis you can keep it as wdio-json-reporter .. it will still be picked up when having reporter setup like reporter: ['json'].

fijijavis commented 5 years ago

When I publish to NPM is there any problem publishing as @wdio/json-reporter instead of wdio-json-reporter?

@fijijavis you can keep it as wdio-json-reporter .. it will still be picked up when having reporter setup like reporter: ['json'].

Sure... I was thinking more about v4 users who might have the dependency setup in their package.json to accept major version changes. I guess I could just add something to the README that says which versions of this package are compatible with v4 / v5 of WDIO

christian-bromann commented 5 years ago

Yeah .. and make sure to make a major version bump and show which version is suited for v4 and v5 upwards

fijijavis commented 5 years ago

@pmerwin @asheeshsingh2014 I have a branch pushed up with all the changes. Would you be available to pull down the branch and test?

asheeshsingh2014 commented 5 years ago

@fijijavis - I was able to test this with v5. I see wdio-0-0-json-reporter.log being created with a JSON of test results. Similarly wdio-0-1-json-reporter.log for the second browser instance. *-json-reporter.log is being created for every browser instance running the spec. Is this intentional?

As far as I remember this reporter used to spit out a results.json single file with all the tests & suites grouped?

christian-bromann commented 5 years ago

any updates on this?

pmerwin commented 5 years ago

@fijijavis any updates?

fijijavis commented 5 years ago

@pmerwin did you get a chance to test? I’m struggling a bit to get time because my org is still on v4 with no immediate timeline to convert. I’ll try to get time to wrap up this weekend

pmerwin commented 5 years ago

I did Jim! Thanks so much looks great!

{
    "start": "2019-04-04T18:59:45.293Z",
    "end": "2019-04-04T19:00:25.369Z",
    "capabilities": {
        "browserName": "chrome",
        "platformName": "MAC"
    },
    "host": "127.0.0.1",
    "port": 4444,
    "baseUrl": "http://localhost",
    "framework": "mocha",
    "mochaOpts": {
        "timeout": 600000,
        "ui": "bdd",
        "bail": true
    },
    "suites": [{
        "name": "User canary can log in and out using chrome on prod",
        "duration": 38453,
        "start": "2019-04-04T18:59:45.483Z",
        "end": "2019-04-04T19:00:23.936Z",
        "tests": [{
            "name": "user can authenticate through the login page using chrome on prod",
            "start": "2019-04-04T18:59:47.647Z",
            "end": "2019-04-04T18:59:55.432Z",
            "duration": 7785,
            "state": "passed"
        }, {
            "name": "can expire access token through auth service using chrome on prod",
            "start": "2019-04-04T18:59:55.433Z",
            "end": "2019-04-04T18:59:55.777Z",
            "duration": 344,
            "state": "passed"
        }, {
            "name": "should refresh the accessToken value in cbtnuggets_user cookie and local storage using chrome on prod",
            "start": "2019-04-04T18:59:55.778Z",
            "end": "2019-04-04T19:00:05.170Z",
            "duration": 9392,
            "state": "passed"
        }, {
            "name": "can open leader board page while logged in and see data using chrome on prod",
            "start": "2019-04-04T19:00:05.171Z",
            "end": "2019-04-04T19:00:06.419Z",
            "duration": 1248,
            "state": "passed"
        }, {
            "name": "can open trainer page and see data logged in using chrome on prod",
            "start": "2019-04-04T19:00:06.419Z",
            "end": "2019-04-04T19:00:08.272Z",
            "duration": 1853,
            "state": "passed"
        }, {
            "name": "can logout as a user using chrome on prod",
            "start": "2019-04-04T19:00:08.273Z",
            "end": "2019-04-04T19:00:11.036Z",
            "duration": 2763,
            "state": "passed"
        }, {
            "name": "can log back in as a user using chrome on prod",
            "start": "2019-04-04T19:00:11.037Z",
            "end": "2019-04-04T19:00:22.866Z",
            "duration": 11829,
            "state": "passed"
        }, {
            "name": "can open trainer page and see data logged out using chrome on prod",
            "start": "2019-04-04T19:00:22.866Z",
            "end": "2019-04-04T19:00:23.935Z",
            "duration": 1069,
            "state": "passed"
        }],
        "hooks": [{
            "start": "2019-04-04T18:59:45.483Z",
            "end": "2019-04-04T18:59:47.647Z",
            "duration": 2164,
            "title": "\"before all\" hook",
            "associatedSuite": "User canary can log in and out using chrome on prod"
        }, {
            "start": "2019-04-04T18:59:47.648Z",
            "end": "2019-04-04T18:59:47.648Z",
            "duration": 0,
            "title": "\"before each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T18:59:55.433Z",
            "end": "2019-04-04T18:59:55.433Z",
            "duration": 0,
            "title": "\"after each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T18:59:55.433Z",
            "end": "2019-04-04T18:59:55.433Z",
            "duration": 0,
            "title": "\"before each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T18:59:55.778Z",
            "end": "2019-04-04T18:59:55.778Z",
            "duration": 0,
            "title": "\"after each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T18:59:55.778Z",
            "end": "2019-04-04T18:59:55.778Z",
            "duration": 0,
            "title": "\"before each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:05.171Z",
            "end": "2019-04-04T19:00:05.171Z",
            "duration": 0,
            "title": "\"after each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:05.171Z",
            "end": "2019-04-04T19:00:05.171Z",
            "duration": 0,
            "title": "\"before each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:06.419Z",
            "end": "2019-04-04T19:00:06.419Z",
            "duration": 0,
            "title": "\"after each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:06.419Z",
            "end": "2019-04-04T19:00:06.419Z",
            "duration": 0,
            "title": "\"before each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:08.272Z",
            "end": "2019-04-04T19:00:08.273Z",
            "duration": 1,
            "title": "\"after each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:08.273Z",
            "end": "2019-04-04T19:00:08.273Z",
            "duration": 0,
            "title": "\"before each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:11.036Z",
            "end": "2019-04-04T19:00:11.036Z",
            "duration": 0,
            "title": "\"after each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:11.037Z",
            "end": "2019-04-04T19:00:11.037Z",
            "duration": 0,
            "title": "\"before each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:22.866Z",
            "end": "2019-04-04T19:00:22.866Z",
            "duration": 0,
            "title": "\"after each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:22.866Z",
            "end": "2019-04-04T19:00:22.866Z",
            "duration": 0,
            "title": "\"before each\" hook",
            "associatedSuite": ""
        }, {
            "start": "2019-04-04T19:00:23.935Z",
            "end": "2019-04-04T19:00:23.935Z",
            "duration": 0,
            "title": "\"after each\" hook",
            "associatedSuite": ""
        }]
    }],
    "specs": ["/Users/pmerwin/Projects/qa/WDIO-5/lib-webdriverio-nodejs-v2/lib/specs/canary/canary.user.auth.js"],
    "state": {
        "passed": 8,
        "failed": 0,
        "skipped": 0
    }
}

for anyone else that wants to test, here is what I did:

from inside my vs 5 lib/reop: git clone git@github.com:fijijavis/wdio-json-reporter.git cd wdio-json-reporter/ npm install git checkout v5_conversion git pull cd ../ npm install ./wdio-json-reporter/ --save set reporters: ['json'], in wdio.conf run tests:

wdio-0-0-json-reporter.log created and looks correct

pmerwin commented 5 years ago

Jim @fijijavis , one thing I found so far is, I think these settings are being ignored:

reporterOptions: { outputDir: config.wdio.prefix, combined: true, filename: config.wdio.filename },

I see wdio-0-0-json-reporter.log get created in logDir: ${WDIO_BASE_DIRECTORY}/wdio-logs, but not in outputDir: config.wdio.prefix, and they are not combined?

christian-bromann commented 5 years ago

and they are not combined?

That option won't work anymore because all reporters live in the worker processes

pmerwin commented 5 years ago

@christian-bromann awe all good, we can do some parsing on our own. :)

fijijavis commented 5 years ago

@pmerwin. Thanks for the feedback. Take a look at the README in that branch. I think I gave an example of how to use outputDir with the new reporter format. Let me know if that doesn’t work or that’s what you are already trying. Thanks again

pmerwin commented 5 years ago

Will do thanks @fijijavis !

fijijavis commented 5 years ago

v1.0.0 published to NPM