selenium-side-driver test results json output doesn't contain test suite names if the tests are executed in parallel

🐛 Bug Report

selenium-side-driver test run result JSON output file doesn't contain test suite names if their tests are executed in parallel

To Reproduce

% selenium-side-runner --filter="unhappy" --output-directory=results demo_project_2.side info: Running v2022_demo_project_2.side PASS unhappy_suite/invalid_credential.test.js (56.718s) PASS unhappy_suite/invalid_credential_v2_blah.test.js (64.264s) PASS unhappy_suite/bad_credential.test.js (67.955s)

Test Suites: 3 passed, 3 total Tests: 3 passed, 3 total Snapshots: 0 total Time: 69.524s Ran all test suites. Test results written to: ../results/demo_project_2.json

Below is the content of the JSON file output. The "ancenstorTitles" values are empty lists instead of "unhappy_suite".

{ "numFailedTestSuites": 0, "numFailedTests": 0, "numPassedTestSuites": 3, "numPassedTests": 3, "numPendingTestSuites": 0, "numPendingTests": 0, "numRuntimeErrorTestSuites": 0, "numTodoTests": 0, "numTotalTestSuites": 3, "numTotalTests": 3, "openHandles": [], "snapshot": { "added": 0, "didUpdate": false, "failure": false, "filesAdded": 0, "filesRemoved": 0, "filesRemovedList": [], "filesUnmatched": 0, "filesUpdated": 0, "matched": 0, "total": 0, "unchecked": 0, "uncheckedKeysByFile": [], "unmatched": 0, "updated": 0 }, "startTime": 1666311350146, "success": true, "testResults": [ { "assertionResults": [ { "ancestorTitles": [], "failureMessages": [], "fullName": "invalid_credential", "location": null, "status": "passed", "title": "invalid_credential" } ], "endTime": 1666311408380, "message": "", "name": "/Users/myuser/demo/side-suite-demo_project_2/unhappy_suite/invalid_credential.test.js", "startTime": 1666311351662, "status": "passed", "summary": "" }, { "assertionResults": [ { "ancestorTitles": [], "failureMessages": [], "fullName": "invalid_credential_v2_blah", "location": null, "status": "passed", "title": "invalid_credential_v2_blah" } ], "endTime": 1666311415921, "message": "", "name": "/Users/myuser/demo/side-suite-demo_project_2/unhappy_suite/invalid_credential_v2_blah.test.js", "startTime": 1666311351657, "status": "passed", "summary": "" }, { "assertionResults": [ { "ancestorTitles": [], "failureMessages": [], "fullName": "bad_credential", "location": null, "status": "passed", "title": "bad_credential" } ], "endTime": 1666311419612, "message": "", "name": "/Users/myuser/demo/side-suite-demo_project_2/unhappy_suite/bad_credential.test.js", "startTime": 1666311351657, "status": "passed", "summary": "" } ], "wasInterrupted": false }

Expected behavior

Test run result JSON output file should contain test suite names if theirs tests are executed in parallel.

Project file reproducing this issue (highly encouraged)

{ "id": "c4eb88d3-9c18-4b8e-821b-e6e0dc35925a", "version": "2.0", "name": "demo_project_2", "url": "https://myurl.com/", "tests": [{ "id": "4a7b0901-1431-4aa3-ab5c-c4bd694e5b3b", "name": "invalid_credential", "commands": [{ "id": "d1944b98-8c98-4557-90e5-1ca7caf4fe17", "comment": "", "command": "run", "target": "global_static_vars", "targets": [], "value": "" }, { "id": "203dc4be-6a16-445a-92fe-fc5b04073f42", "comment": "", "command": "store", "target": "${invalid_username}", "targets": [], "value": "username" }, { "id": "b7a77db9-95d4-499d-9f72-6a8e7435df6c", "comment": "", "command": "store", "target": "${invalid_password}", "targets": [], "value": "password" }, { "id": "f6bd6d86-a620-442f-94e8-9b88ec9cd97c", "comment": "", "command": "run", "target": "login", "targets": [], "value": "" }, { "id": "a049f4e2-d993-4c86-93d7-9de878947c51", "comment": "", "command": "assertText", "target": "css=.alert:nth-child(2) > span", "targets": [], "value": "Sorry, we were not able to find a user with that username and password." }] }, { "id": "80b92de5-e26d-4bde-8e3c-2818c9155f9b", "name": "invalid_credential_v2_blah", "commands": [{ "id": "4960df13-04d1-427b-a5dc-4abf303a5583", "comment": "", "command": "run", "target": "global_static_vars", "targets": [], "value": "" }, { "id": "df86218e-4cb8-4383-9c33-257908869790", "comment": "", "command": "store", "target": "${invalid_username}", "targets": [], "value": "username" }, { "id": "94186dfa-3dce-4ee2-acd5-f14aed59e5f5", "comment": "", "command": "store", "target": "${invalid_password}", "targets": [], "value": "password" }, { "id": "20e98354-a899-4557-ba3c-eacdea3bd928", "comment": "", "command": "run", "target": "login", "targets": [], "value": "" }, { "id": "ff591ecb-a6f5-431e-b8b5-16855f1fc126", "comment": "", "command": "assertText", "target": "css=.alert:nth-child(2) > span", "targets": [], "value": "Sorry, we were not able to find a user with that username and password." }] }, { "id": "e778bf8b-566a-4197-a62a-2e3505d8e350", "name": "global_static_vars", "commands": [{ "id": "adb7cb16-eb3a-4f67-90ee-32b0632a5a7f", "comment": "", "command": "store", "target": "user1", "targets": [], "value": "valid_username" }, { "id": "f413f97e-e17d-4248-83fe-d0199503080e", "comment": "", "command": "store", "target": "paswword1", "targets": [], "value": "valid_password" }, { "id": "81ffa638-b2cb-4547-a45e-2f1fb3172821", "comment": "", "command": "store", "target": "blah", "targets": [], "value": "invalid_username" }, { "id": "27f3f303-2d7f-474b-9d56-a3ca412d6d06", "comment": "", "command": "store", "target": "blah", "targets": [], "value": "invalid_password" }] }, { "id": "b5d3d691-f830-4e81-a86f-83175610fd64", "name": "login", "commands": [{ "id": "a5f1ac7a-8a1f-441e-ba7e-74378d94db07", "comment": "", "command": "open", "target": "login/auth", "targets": [], "value": "" }, { "id": "fd38906d-c250-4c78-a8e3-898ee1d179bb", "comment": "", "command": "setWindowSize", "target": "1588x1113", "targets": [], "value": "" }, { "id": "e670d3a0-28a4-41f8-a848-7817f68d7472", "comment": "", "command": "click", "target": "id=username", "targets": [ ["id=username", "id"], ["name=j_username", "name"], ["css=#username", "css:finder"], ["xpath=//input[@id='username']", "xpath:attributes"], ["xpath=//input", "xpath:position"] ], "value": "" }, { "id": "720efb69-383a-4d5b-8079-5bbdd16a6bad", "comment": "", "command": "type", "target": "id=username", "targets": [ ["id=username", "id"], ["name=j_username", "name"], ["css=#username", "css:finder"], ["xpath=//input[@id='username']", "xpath:attributes"], ["xpath=//input", "xpath:position"] ], "value": "${username}" }, { "id": "1cca6d0f-091f-48b8-938e-aaac5b1d7518", "comment": "", "command": "click", "target": "name=j_password", "targets": [ ["name=j_password", "name"], ["css=.form-label-group:nth-child(3) > .form-control", "css:finder"], ["xpath=//input[@name='j_password']", "xpath:attributes"], ["xpath=//div[3]/input", "xpath:position"] ], "value": "" }, { "id": "513f52df-405f-4ca0-a612-89cb36bb4372", "comment": "", "command": "type", "target": "name=j_password", "targets": [ ["name=j_password", "name"], ["css=.form-label-group:nth-child(3) > .form-control", "css:finder"], ["xpath=//input[@name='j_password']", "xpath:attributes"], ["xpath=//div[3]/input", "xpath:position"] ], "value": "${password}" }, { "id": "a23dd99e-ee6f-47ed-ab08-a7696d5274ae", "comment": "", "command": "click", "target": "css=.btn", "targets": [ ["css=.btn", "css:finder"], ["xpath=//button[@type='submit']", "xpath:attributes"], ["xpath=//div[4]/div/button", "xpath:position"], ["xpath=//button[contains(.,'Login')]", "xpath:innerText"] ], "value": "" }] }, { "id": "537d47c1-8c8e-42eb-95e9-c4f3a666d5d5", "name": "logout", "commands": [{ "id": "3e96ed6c-3ff3-40d1-9eeb-b273de45a944", "comment": "", "command": "click", "target": "css=.user-image", "targets": [ ["css=.user-image", "css:finder"], ["xpath=//div[@id='logoutDD']/div", "xpath:idRelative"], ["xpath=//div[5]/div/div", "xpath:position"] ], "value": "" }, { "id": "4e3f3b66-bb2f-466a-b1ca-24b9d48cd366", "comment": "", "command": "click", "target": "css=.dropdown-item:nth-child(4)", "targets": [ ["css=.dropdown-item:nth-child(4)", "css:finder"], ["xpath=//div[@id='root']/div/header/div/div/div[4]/div[5]/ul/li[4]", "xpath:idRelative"], ["xpath=//li[4]", "xpath:position"], ["xpath=//li[contains(.,'Log Out')]", "xpath:innerText"] ], "value": "" }] }, { "id": "320cc669-a912-4688-8fec-93690c40dffc", "name": "bad_credential", "commands": [{ "id": "5b78af93-eab8-4c35-8fe8-704dd313d942", "comment": "", "command": "run", "target": "global_static_vars", "targets": [], "value": "" }, { "id": "dd063b90-6b5e-4571-8896-73ad25af48a2", "comment": "", "command": "store", "target": "${invalid_username}", "targets": [], "value": "username" }, { "id": "c9bf47e7-6b5d-4d13-9894-4a8eba199c79", "comment": "", "command": "store", "target": "${invalid_password}", "targets": [], "value": "password" }, { "id": "6efea7f0-206e-49b6-8979-ad3c3f812a75", "comment": "", "command": "run", "target": "login", "targets": [], "value": "" }, { "id": "5e98f221-08fb-47e1-ba82-2569433575b4", "comment": "", "command": "assertText", "target": "css=.alert:nth-child(2) > span", "targets": [], "value": "Sorry, we were not able to find a user with that username and password." }] }, { "id": "9c85a005-3f52-4249-8030-5b7c17390c2b", "name": "good_credential", "commands": [{ "id": "cb19706f-899e-4cf6-8fcb-08a594dcb34c", "comment": "", "command": "run", "target": "global_static_vars", "targets": [], "value": "" }, { "id": "bd994f5d-b5f6-4b95-9fc0-f2d2a104b075", "comment": "", "command": "store", "target": "${valid_username}", "targets": [], "value": "username" }, { "id": "7eb2a654-91b3-44d2-bf16-c7d8389174cf", "comment": "", "command": "store", "target": "${valid_password}", "targets": [], "value": "password" }, { "id": "02ea0076-c003-47b1-9686-871d24d9b1ed", "comment": "", "command": "run", "target": "login", "targets": [], "value": "" }, { "id": "bbc36519-4c33-4543-b1bb-4df4d2319198", "comment": "", "command": "run", "target": "logout", "targets": [], "value": "" }] }], "suites": [{ "id": "55446e0b-1b88-4fe5-870d-e49e55d4f506", "name": "unhappy_suite", "persistSession": false, "parallel": true, "timeout": 300, "tests": ["320cc669-a912-4688-8fec-93690c40dffc", "4a7b0901-1431-4aa3-ab5c-c4bd694e5b3b", "80b92de5-e26d-4bde-8e3c-2818c9155f9b"] }, { "id": "0a3af420-9b07-48f2-9818-014444e1f417", "name": "awesome_suite", "persistSession": false, "parallel": false, "timeout": 300, "tests": ["9c85a005-3f52-4249-8030-5b7c17390c2b"] }, { "id": "ab66ac7f-b3df-4aa2-986e-d8ddbd4c54dc", "name": "common_suite", "persistSession": false, "parallel": false, "timeout": 300, "tests": ["b5d3d691-f830-4e81-a86f-83175610fd64", "537d47c1-8c8e-42eb-95e9-c4f3a666d5d5", "e778bf8b-566a-4197-a62a-2e3505d8e350"] }], "urls": ["https://myurl.com/", "plugins": [] }

Environment

OS: MacOS Monterey 12.6 Selenium IDE Version: v3.17.0 Selenium SIDE Runner Version: 4.0.0-alpha.15 Node version: v18.10.0 ChromeDriver version: 106.0.5249.61 Browser: Browser Version: 106.0.5249.61

SeleniumHQ / selenium-ide