side-code-export misaligns command's comments

[matewilk]

🐛 Bug Report

I'm having a standard *.side file containing clicks, input fields and dropdown selections, the file also contains comments so just to depict how part of it looks like:

...
{
    "id": "f58a7a8e-316b-4255-a6c3-112cdccaaf0e",
    "comment": "Click gender",
    "command": "click",
    "target": "id=gender",
    "targets": [
      ["id=gender", "id"],
      ["css=#gender", "css:finder"],
      ["xpath=//select[@id='gender']", "xpath:attributes"],
      ["xpath=//form[@id='contactus']/div[3]/div/label/select", "xpath:idRelative"],
      ["xpath=//select", "xpath:position"]
    ],
    "value": ""
  }, {
    "id": "d052f7af-d01f-4d84-8337-88f5e42ef8ea",
    "comment": "Click select",
    "command": "select",
    "target": "id=gender",
    "targets": [],
    "value": "label=My Business!"
  }, {
    "id": "ef17f22a-8de2-4dcc-b6bb-ecbd6592f23c",
    "comment": "Click id=blue",
    "command": "click",
    "target": "id=blue",
    "targets": [
      ["id=blue", "id"],
      ["css=#blue", "css:finder"],
      ["xpath=//input[@id='blue']", "xpath:attributes"],
      ["xpath=//form[@id='contactus']/div[4]/div/input[2]", "xpath:idRelative"],
      ["xpath=//input[2]", "xpath:position"]
    ],
    "value": ""
}
...

I also have a command emitSelect (among other commands) that is multi line command:

const emitSelect = async (selectElement: string, option: string) => {
  const commands = [
    { level: 0, statement: `{` },
    {
      level: 1,
      statement: `const dropdown = await $webDriver.wait(until.elementLocated(${await location.emit(
        selectElement
      )}))`,
    },
    {
      level: 1,
      statement: `await dropdown.findElement(${await selection.emit(
        option
      )}).click()`,
    },
    { level: 0, statement: `}` },
  ];
  return Promise.resolve({ commands });
};

When I run side-code-export with my set of commands, the comments are being applied properly at first for one line commands, but the comments are becoming misaligned as soon as the code-export encounters a multi line command.

This is how it turns out:

// Generated by Selenium IDE
// dependencies go here

// suite declaration Form Select Checkbox
const By = $selenium.By;
const until = $selenium.until;
const vars = new Map();

// beforeEach
  // test declaration Selenium test page
    // Open window
    await $webDriver.get("https://automationintesting.com/selenium/testpage/")
    // Set window size
    await $webDriver.manage().window().setRect({ width: 1680, height: 940 })
    // Press first name
    await $webDriver.wait(until.elementLocated(By.id("firstname"))).click()
    // Type first name
    await $webDriver.wait(until.elementLocated(By.id("firstname"))).sendKeys("Bob")
    // Type surname
    await $webDriver.wait(until.elementLocated(By.id("surname"))).sendKeys("Sapp")
    // Click gender
    await $webDriver.wait(until.elementLocated(By.id("gender"))).click()
    // Click select // <<---- this is still OK
{
// Click id=blue // <<---- this should be after the closing bracket
  const dropdown = await $webDriver.wait(until.elementLocated(By.id("gender")))
  // Click textarea // <<---- this should be above the textarea click command
  await dropdown.findElement(By.xpath("//option[. = 'My Business!']")).click()
  // Type into textarea // <<---- this should be just above textarea sendKeys (you get the point)
}
await $webDriver.wait(until.elementLocated(By.id("blue"))).click()
await $webDriver.wait(until.elementLocated(By.css("textarea"))).click()
await $webDriver.wait(until.elementLocated(By.css("textarea"))).sendKeys("more stuff")
{
  const dropdown = await $webDriver.wait(until.elementLocated(By.id("continent")))
  await dropdown.findElement(By.xpath("//option[. = 'Europe']")).click()
}
  // terminating keyword
// terminating keyword

I've tried searching for where this belongs but couldn't find the appropriate lines of code responsible for that. I'm happy to investigate and create a PR with a fix, and I'd love to be pointed out in the right direction on where to start my investigation.

To Reproduce

Run the file against code-export-javascript-mocha and you'll notice the behaviour is the same there.

Expected behavior

Comments are added before every command's output, not in between the lines of a command, but between commands (hope it makes sense)

Project file reproducing this issue (highly encouraged)

here is the project *.side file: https://github.com/matewilk/js-newrelic-synthetics/blob/main/projects/form-select-checkbox.side

You can clone and download this project https://github.com/matewilk/js-newrelic-synthetics to reproduce the issue or use the code-export-javascript-mocha package from this repo mentioned above.

Run:

• npm install
• npm run build
• npm run build:code

This will create the .js output file under /tests directory with the described issue.

[matewilk]

Ok, so I've been digging for two days and learning the codebase and with the current architecture this problem is very hard to solve, I'll just summarise here what I've found so far.

The function responsible for merging the commandBodies with originTracing is here

The problem lies in the incorrect alignment and missing comments in the output of the renderCommands function. The function is supposed to iterate over an array of commands (commandBodies) and an array of corresponding comments (originTracing), and generate a string (result) that combines the commands and their descriptions. However, the function is not correctly aligning the comments with the corresponding commands, and it's also missing some comments.

Specifically, the issues are:

1. Misalignment of comments: The comments (origin tracing) are not correctly aligned with the corresponding commands. For example, the comment "// Click id=blue" is placed before the block of code that it should be associated with, instead of being placed before the command await $webDriver.wait(until.elementLocated(By.id("blue"))).click().

2. Missing comments: The last comment "// Select from dropdown" is completely missing from the output. This might be due to the fact that the originTracing array has fewer elements than the commandBodies array. If the function assumes that the two arrays have the same length, it might not handle the case where the originTracing array is shorter.

The challenge is to modify the renderCommands function to correctly align the comments with the commands and handle the case where the originTracing array has fewer elements than the commandBodies array.

Now to elaborate further having the original bug issue in mind this is how the commandBodies and originTracing arrays look like:

commandBodiex

[
  "await $webDriver.get(\"https://automationintesting.com/selenium/testpage/\")",
  "await $webDriver.manage().window().setRect({ width: 1680, height: 940 })",
  "await $webDriver.wait(until.elementLocated(By.id(\"firstname\"))).click()",
  "await $webDriver.wait(until.elementLocated(By.id(\"firstname\"))).sendKeys(\"Bob\")",
  "await $webDriver.wait(until.elementLocated(By.id(\"surname\"))).sendKeys(\"Sapp\")",
  "await $webDriver.wait(until.elementLocated(By.id(\"gender\"))).click()",
  {
    level: 0,
    statement: "{",
  },
  {
    level: 1,
    statement: "const dropdown = await $webDriver.wait(until.elementLocated(By.id(\"gender\")))",
  },
  {
    level: 1,
    statement: "await dropdown.findElement(By.xpath(\"//option[. = 'My Business!']\")).click()",
  },
  {
    level: 0,
    statement: "}",
  },
  "await $webDriver.wait(until.elementLocated(By.id(\"blue\"))).click()",
  "await $webDriver.wait(until.elementLocated(By.css(\"textarea\"))).click()",
  "await $webDriver.wait(until.elementLocated(By.css(\"textarea\"))).sendKeys(\"more stuff\")",
  "",
  "",
  "",
  {
    level: 0,
    statement: "{",
  },
  {
    level: 1,
    statement: "const dropdown = await $webDriver.wait(until.elementLocated(By.id(\"continent\")))",
  },
  {
    level: 1,
    statement: "await dropdown.findElement(By.xpath(\"//option[. = 'Europe']\")).click()",
  },
  {
    level: 0,
    statement: "}",
  },
]

originTracing

[
  "// Open window",
  "// Set window size",
  "// Press first name",
  "// Type first name",
  "// Type surname",
  "// Click gender",
  "// Click select",
  "// Click id=blue",
  "// Click textarea",
  "// Type into textarea",
  "",
  "",
  "",
  "// Select from dropdown",
]

The loop does not correctly match those two arrays and the problem is really hard to solve with the current structure due to a lot of edge cases.

Posible solutions

The correct approach (a lot of work)

I think the better approach would be to save those into objects and iterate over objects and compare its keys which would represent where a comment/command should be render. This would require re-architecting possibly large parts of the codebase.

A hacky solution (little work)

At lest for my uses case would be to squeeze multi line commands into a single line command so for example instead of:

const emitSelect = async (selectElement: string, option: string) => {
  const commands = [
    { level: 0, statement: `{` },
    {
      level: 1,
      statement: `const dropdown = await $webDriver.wait(until.elementLocated(${await location.emit(
        selectElement
      )}))`,
    },
    {
      level: 1,
      statement: `await dropdown.findElement(${await selection.emit(
        option
      )}).click()`,
    },
    { level: 0, statement: `}` },
  ];
  return Promise.resolve({ commands });
};

I'd have:

const emitSelect = async (selectElement: string, option: string) => {
  const statements = [
    `{`,
    `const dropdown = await $webDriver.wait(until.elementLocated(${await location.emit(
      selectElement
    )}))`,
    `await dropdown.findElement(${await selection.emit(option)}).click()`,
    `}`,
  ];

  const commands = [{ level: 0, statement: statements.join("\n") }];

  return Promise.resolve({ commands });
};

this is not ideal but would ensure that the number of commands and comments in both array are the same so the current codebase aligns them as expected

[toddtarsi]

There this is! I saw your message from two days ago, I'm so sorry I couldn't find the thread. Thank you for diving the hell out of this and bringing back all this info and context. Invaluable.

I completely agree. This needs to be rewritten. Currently commands have like 3 or 4 different syntax permutations they come in as, and it complicates the hell out of this. Converting this to typescript was a HILARIOUS mess of spaghetti. This is another reason I think each command should really be in a file like (folder structure is entirely made up and not indicative of anything) formats/new-relic-js/commands/click.js. This way we always start from a string literal, but also it gets us closer to having a nicer time in IDEs editing formats and such

[toddtarsi]

@matewilk - Ok this doesn't help you at all but there's now a command line arg to take screenshots on failure. No it doesn't help your case at all, but its pretty nice if you're using this in a CI.

[toddtarsi]

I wouldn't bother you with this but I don't do a good job of maintaining a changelog, and I want to tell somebody haha

[matewilk]

I'm with you man, thanks for sharing 💫 😎