mobile-dev-inc / maestro

Painless Mobile UI Automation
https://maestro.mobile.dev/
Apache License 2.0
5.66k stars 255 forks source link

[Feature Request] Incorporate .maestro directory structure in JUnit report #1418

Open saihemanth121 opened 1 year ago

saihemanth121 commented 1 year ago

Is your feature request related to a problem? Please describe. I have 50+ maestro flows in my Android project which we have organized as recommended here. We use TeamCity which can parse JUnit reports and show tests organized by suite, package, class etc. But the maestro generated JUnit report does not include the directory structure, so the report is basically a flat list of tests executed.

Describe the solution you'd like If the directory structure of the flows can be incorporated somehow in the JUnit report, say using class or package, then the imported test report will be easier to navigate after parsing.

Describe alternatives you've considered Only alternative that I could come up with is to prefix the flow names with the directory structure, so that we can sort the tests for readability.

Additional context TeamCity test report page: https://www.youtube.com/watch?v=LKJjcBJT1k0

langtind commented 1 year ago

Related, but unsure if it should be included in this feature request: Currently, all fields in the report contain the same information. flowname is used in id, name, and classpath. Would it be possible to change one of these fields to contain the filename of the actual test?

Our use case involves parsing the report of a test run to prepare for a rerun of only the failed tests. Although we have a working solution, I believe it would be more intuitive to directly find the filename from the report instead of having to use grep on the test directory to obtain it.

test_file=$(grep -rl "Flowname - A flow we want to address from the report" ./tests)
echo $test_file
amanjeetsingh150 commented 1 year ago

Hey @saihemanth121 thanks for reaching out! Is it going to help if each junit report will be generated in the same structure like your flows are already structured in workspace and with prefix of flow name?

saihemanth121 commented 1 year ago

@amanjeetsingh150 thanks for replying. I don't think it'll help in my case: the location of the JUnit report will be ignored even if we import multiple reports. The content of the JUnit report needs to indicate the test class/package.

amanjeetsingh150 commented 1 year ago

The content of the JUnit report needs to indicate the test class/package.

Do you mean flow name right? Which class/package are you referring to?

saihemanth121 commented 1 year ago

I meant classname param in the JUnit report. It'd help if classname is equal to the path of the flow file (ex: folder1.folder2.flowName if flowName.yaml is in .maestro/folder1/folder2). Currently the id, name and classname in the report are just the flow file name.