Performance csv and json files are all blank | Using webdriverIO latest version viz. 7.1.2 & @wdio/cucumber-framework

[vdrulerz]

My tests are all passing and I am just using below lines in my step definition but after showing success status it throws performance total error that could not find the log file.....not sure how to generate this log file and why has it not generated by default ?? Kindly help me on investigating this issue. Thanks in advance !

[tzurp]

@vdrulerz, please check version 1.0.9-1 (beta) here: https://www.npmjs.com/package/wdio-performancetotal-service/v/1.0.9-1 and tell me if it solved the problem. To install the beta write: npm install wdio-performancetotal-service@1.0.9-1

[vdrulerz]

Hi @tzurp .....behavior is still the same nothing has changed I can confirm that I have successfully installed 1.0.9-1 but after running the tests it is showing blank json, blank csv and the same error message which I shared above. Do you want to see my tsconfig.json file or anything you want for investigating this issue. I am simply using the example which is there on webdriverio official docs but it is not showing any metrics :( I think you should update WDIO packages to latest and then test this package.

[tzurp]

@vdrulerz did it work for you for WDIO v6.x ? Which node.js version you're using? Can you please collect all the Performance-Total messages from the log for a single test run and add them here?

[vdrulerz]

Thanks for your prompt response ! I am using node V 12.13.0 & typescript version 4.2

And I dint check in v6.x reason being few days back only I got to see about this service in WDIO docs so thought of using it for calculating the performance time for loading a page.

@tzurp From where can I grab performance-total messages ....can you please guide me so that I can provide you this sooner rather than later

[tzurp]

@vdrulerz, In the IDE terminal, where all your logs are written into, search (Ctrl+F) for 'Performance-Total' and copy all the relevant lines. There shouldn't be much of them. In the screenshot you attached above I can see the "Performance-Total error: readFile failed..." but there are more related messages that are required.

[vdrulerz]

@tzurp I have validated already by doing Ctrl+F, my log level is Info

And the first match was "Performance-Total error: readFile failed" this only.....do you have any sample working repo which I refer and run that on my system just to check if I am missing anything

[tzurp]

@vdrulerz If you're using the default settings of PerformanceTotal you should have this path: '/performance-results/performance-log.txt'. Can you verify that the file 'performance-log.txt' is existing on your machine? Can you confirm that the path: '/performance-results' was created on your machine? Another thing: Can you find in the log the line: 'Performance-Total: flushing to file: path/to/performance-log.txt' ?

[vdrulerz]

[vdrulerz]

I think this is the only problem that performance-log.txt file is not there

[vdrulerz]

Just seeing these two files which are all blank

[vdrulerz]

Even /performance-results/ this folder is not present :(

[tzurp]

@vdrulerz since I can't reproduce your results, the best way to try to solve this is by sharing your repo so I'll be able to analyze the problem. If you can't share it I suggest you create a new small project as described here: https://webdriver.io/docs/gettingstarted#installation, and install the service. I upgraded the main version of PerformanceTotal so please don't use the Beta version.

[vdrulerz]

Sure, I will share the repo with you. Appreciate your help !

It will also be helpful if you can create one sample example repo for others to refer to it with the name of "performance-total-example". Which would simply be using your service by calling any website url like webdriverIO

I can take a stab at it if you want me to ??

[tzurp]

Here is the example repo: https://github.com/tzurp/performance-total-demo

[vdrulerz]

I will simply use this and run it as is...then will change the framework to cucumber and try it out.

this performance-results folder itself was not getting generated on my system .....which was causing all the problem I think.

Anywyz please keep this issue open and I will update you with more details....thanks again !

[vdrulerz]

@tzurp It's a valid issue and it's coming in your shared repo too, issue is with the cucumber framework. I would suggest you to update below in your repo and you will be able to reproduce the same

• Install @wdio/cucumber-framework
• Add its type in tsconfig.json
• Simply add one feature file and one step-def file
• Change framework in config file from mocha to 'cucumber' & add cucumberOpts (require)
• Now run the tests, you would notice that stats are not coming

The moment I switch it back to Mocha, everything works well

Hence I am going to update the issue title to 'Cucumber' framework so that there won't be any confusion

[vdrulerz]

@tzurp - Can you please acknowledge the issue that for cucumber framework json is coming as blank array ??

Did you try replicating the same ??

[tzurp]

@vdrulerz thanks for the fruitful inquiry. It seems that when using Cucumber framework the service hooks are skipped (at least for wdio v7.x). If so, it seems to be a wdio bug. Can you please upgrade to the latest wdio version (currently is v7.2.1) and if you still encounter the problem you could open an issue in wdio here. Though, there is a workaround that let you use PerformanceTotal as a non-service library, I don't support it officially, and I think that using it as a service is much easier. In addition fixing the wdio bug would contribute a lot to the community.

[vdrulerz]

@tzurp Thanks for acknowledging the same, u know it would be great if you log an issue there and tag all our conversation. U might already be knowing where exactly the code is getting skipped so u will log it with more details than me and I agree if it's a wdio cucumber library bug then it should be fixed at that point only.

Many people are using cucumber so if this can be fixed sooner rather than later then the usage of this service can get better down the line.

[erwinheitzman]

Hi there @tzurp neat service you've got. @vdrulerz asked me if I could take a quick look at this conversation as a maintainer for the webdriverio project.

I have taken a quick look at the service and is my assumption correct when I say that you feel that the beforeTest and afterTest hooks are not triggered when using Cucumber?

[tzurp]

Hi @erwinheitzman Thanks! Yes, it seems that when using the Cucumber framework only the after hook is triggered for the service, and the 'afterTest' and 'beforeTest' are not.

[erwinheitzman]

I'm not sure if this is intentional but for Cucumber the beforeScenario and afterScenario are used instead which explains this behavior. I do feel that this worked differently in webdriverio v6 but I would have to verify this.

[vdrulerz]

@erwinheitzman Does that mean in service file below, cucumber related scenario conditions would need to be added ?? and only then for cucumber framework this service would work ??

https://github.com/tzurp/performance-total/blob/master/src/service.ts

@tzurp

[erwinheitzman]

@vdrulerz If it is intended, yes. If not, we have an issue in webdriverio v7 for which I will then create an issue accordingly.

[vdrulerz]

@erwinheitzman Thanks.....do you want me to check this service in webdriver V6 with cucumber framework then ?? what if for V6 also this would behave the same ??

I mean then the issue would still be at WDIO side or at this service side ?? not sure :(

[erwinheitzman]

@vdrulerz I'm already on it

[erwinheitzman]

@vdrulerz @tzurp i just verified that this behavior is the same in v6 so in order to fix this when using Cucumber I feel it would be good to isolate the logic and call this in both the beforeText and beforeScenario hook aswell as the afterTest and afterScenario hook.

EDIT: created a small PR for this that should fix the issue (untested): https://github.com/tzurp/performance-total/pull/16

@tzurp if you want to improve the quality of your service and get some pointers, feel free to reach out and we can have a chat about some things you could do to improve the readability, testability and maintainability of your code

[vdrulerz]

@erwinheitzman Thanks is really a very small word for all the efforts you have put in to go through my comments in gitter and make the changes here.

You truly are setting a great example....thanks for everything ! cheers !

[tzurp]

@erwinheitzman Thanks! IMO this should be added to the documentations of the custom worker service.

[erwinheitzman]

I have just pushed a small change because the status that Cucumber provides is also different from how it's passed from Mocha/Jasmine.

I have taken the enum from the code in webdriverio for clarity.

@tzurp I think that's a great idea. Would you mind adding this documentation in a PR?

[tzurp]

@erwinheitzman Yes right after we'll test the fix (:

[vdrulerz]

@tzurp can you deploy a beta version from Erwin's PR so that I can install that beta version and test the fix??

[tzurp]

Sure. I'll let you know when it is ready.

[tzurp]

@vdrulerz can you please check the updated version? Type: npm install wdio-performancetotal-service@1.0.10-0 --save-dev

[vdrulerz]

Hi @tzurp We can say it is mostly fixed but in the logs I am seeing the wrong test status for cucumber. So I am putting the details right here.

If you check above screenshot, with mocha I ran the tests on 19th March and isTestPassed flag was TRUE, but today with cucumber it generated the logs in the text file however isTestPassed status was set as FALSE even when my tests were all passed.

Can you please check how you are setting this flag ?? looks like for cucumber framework this flag is not setting correctly

[tzurp]

@vdrulerz try this: npm install wdio-performancetotal-service@1.0.10-1 --save-dev

[tzurp]

@erwinheitzman, can I contact you directly via LinkedIn or other way?

[vdrulerz]

[vdrulerz]

Behavior is still the same with 1.0.10-1 version

[erwinheitzman]

You can message me on Gitter or on LinkedIn.

Do you want me to take a look at this or do you want to look into this @tzurp ?

[tzurp]

Sorry my mistake. This is the one: npm install wdio-performancetotal-service@@1.0.10-2 --save-dev

[vdrulerz]

@tzurp Yeah with 1.0.10-2 this seems to be fixed

Can you also guide me how to populate this chart below ???

[vdrulerz]

@tzurp Feel free to mark this issue as closed because the original problem is resolved now

Once you get some time do guide me how to populate the metrics above ??

[tzurp]

@vdrulerz Just open the *.csv file using Excel (or other spreadsheet app) and generate a graph on the obtained data. At first I thought to generate it in the service plugin, but I realized how easy and productive is to use a generic tool like Excel (; (BTW if you need help with graphing the SEM just ask me how). From your side, can I close the issue?

[vdrulerz]

Yes please @tzurp this issue you can close

Thanks for all the follow up from last few days...indeed it was super helpful...if I will have any further query I will ask you using "How"

And yeah that metric could definitely be next major update of this service 2.0 performance-total-service :)

[vdrulerz]

@erwinheitzman Thanks again for all your help 👍

[narkanag]

Is it work for android native/Hybrid App