Selenium IDE tests failing in Selenium Side Runner

[ghost]

Banner9Test_SeleniumHQ.side.zip

🐛 Bug Report

A clear and concise description of what the bug is.

I have Selenium IDE test scripts (.side files) that I've created; and I'm running them from the command line, using selenium-side-runner. These test scripts run without error in Selenium IDE, but when I run them using selenium-side-runner they fail.

Here are some typical errors from the failed side-runner execution: TimeoutError: Waiting for element to be located By(css selector, *[id="inp\:id"]) Wait timed out after 30114ms TimeoutError: Waiting for element to be located By(xpath, //i[@class='ba ba-close']) Wait timed out after 45048ms TimeoutError: Waiting for element to be located By(css selector,

KEY_BLOCK_CANVAS_keyblckAidyCodeLbt > .ui-button-icon-primary)

Wait timed out after 45141ms

For security reasons, I cannot include any code here

To Reproduce

Run the .side file from command line using selenium-side-runner

Expected behavior

The .side file runs without error

Project file reproducing this issue (highly encouraged)

Your form here will not accept .side files. I've tried to paste mine in here, but it will not take it. You're expecting a text, Word, image, pdf, or zipped file. What am I supposed to do???!

Environment

OS: macOS Mojave 10.14.6 Selenium IDE version: 3.17.0 selenium-side-runner version: 3.16.0 Node version: 10.16.0

Browser: Browser version: Firefox 74.0 (64-bit)

My email: david.mcglochlin@wgu.edu

[tourdedave]

@dmcglochlin you can zip up the side file and attach that.

[ghost]

Thanks for the suggestion to zip the .side file and send it.

[Danc2050]

I also have this issue! Thanks for sharing this @dmcglochlin and sharing your .side file. I would love for this to get fixed. I am running the following versions of software:

OS: Windows 10 Selenium IDE Version: 3.17.0 Chrome driver: 84.0.4147.30 Chrome: 84.0.4147.125 Selenium-side-runner: 3.17.0 Node: 12.18.3

[yongchoo]

I am also having a similar problem with IDE & selenium-side-runner: on Windows10 ElementClickInterceptedError: Element is not clickable at point (150,593) because another element obscures it IDE = 3.17.0, Firefox = 81.0, geckodriver = 1.20.0, Selenium-side-runner: 3.17, node: 12.18.0

Same issue with chrome too... chromedriver: 85.0.1 2ndProject.zip

[yongchoo]

After doing some more 'experiments' I was able to run selenium-side-runner by: option 1: manually changed setWindowSize in the .side file to the max for my screen (1936x1056) option 2: in the IDE, deleted any entries with "out-line-danger" for the .side test file, then saved this file So for my case, IDE generates additional 'test entries' which are not compatible with selenium-side-runner. Perhaps this will hep

[yongchoo]

I forgot to mention that my test url was https://react-redux.realworld.io

[jackrabbithanna]

I also have experienced this or more problems. It appears that "wait for element visible" commands are ignored? I created a simple test, works fine in the IDE, exported it, but selenium-side-runner reports "Your test suite must contain at least one test." I edit the .side fail, plain as day there is a "tests" section in the json. Other people can take the .side files and run in Selenium IDE on other computers

I already was setting the window size to fit my computer. I don't see any entries about "out-line-danger" .. whatever that is.

Is this tool supposed to work with any tests created in Selenium IDE Chrome extension?

Selenium Side Runner 3.17 Node 10 Chrome 87 Whatever version of chromedriver which came automatically running this command: npm install -g chromedriver

[florian-asche]

Same issue. Happy to provide testfiles if needed.

[slovatt]

I'm having the same problem.

My tests run fine in Chrome, but fail in the Command Line (via Jenkins):

TimeoutError: Waiting for element to be located By(link text, linkName) Wait timed out after 15151ms

I've even tried adding pauses (even though it's bad practice!), refreshing the page, etc, but the element still isn't found.

[djsunderland]

This is affecting our use of Selenium IDE as well.

Is there any ETA on a fix for this?

[5er9e1]

two years passed but side runner and Selenium IDE still incompatible.

I tried to wait before element present, wait before element visible, I added pause after every step manually and still getting new errors. Even 5 steps can't be passed without debugging and manual script editing. It is easier and faster to create own test using usual programming language.

element is not attached to the page document, element not interactable, TimeoutError: Waiting for element to be located By(css selector, ...) and other endless errors.

Also tried to export test to pytest and it failing too:

actions.move_to_element(element, 0, 0).perform()
TypeError: ActionChains.move_to_element() takes 2 positional arguments but 4 were given

Completely not usable for automation tasks.

PS Of course script perfectly working inside Selenium IDE

[toddtarsi]

@5er9e1 - Mid-April friend, please come back in April. This is where I'm working on v4, and I'm aiming to get a beta out then.

https://github.com/SeleniumHQ/selenium-ide/pull/1258

The things I don't have wired up yet are right click context menus, a fair amount of test coverage, and plugin architecture. Lots of little edge cases to fine tune too, but the core of what you're asking for is there.

If you don't care about those elements, feel free to check out the branch and use v4. The build commands are very simple:

yarn; yarn build; yarn start;

You'd be the third person to use it, but this gets about three hours a night from me. It will be ready soon 🙂

[toddtarsi]

Just also wanted to note, if you're cool with waiting until mid-April, those cases will be better handled and there will be a pre-built binary that you can just download and use, no build commands needed.

[Thadiusdog]

I also have experienced this or more problems. It appears that "wait for element visible" commands are ignored? I created a simple test, works fine in the IDE, exported it, but selenium-side-runner reports "Your test suite must contain at least one test." I edit the .side fail, plain as day there is a "tests" section in the json. Other people can take the .side files and run in Selenium IDE on other computers

I already was setting the window size to fit my computer. I don't see any entries about "out-line-danger" .. whatever that is.

Is this tool supposed to work with any tests created in Selenium IDE Chrome extension?

Selenium Side Runner 3.17 Node 10 Chrome 87 Whatever version of chromedriver which came automatically running this command: npm install -g chromedriver

Been a while since I used Selenium IDE and found out that if you get this error while trying to run selenium-side-runner "Your test suite must contain at least one test." It might be because you haven't added your any of your Tests to the Test Suite in Selenium IDE.

[karasma3]

hello @toddtarsi i just wanted to say, that right click we implemented this way: execute script command with

const el = document.querySelector('CSS_SELECTOR_GOES_HERE');

const eventContextMenu = new MouseEvent('contextmenu', {
    bubbles: true,
});

el.dispatchEvent(eventContextMenu);

it ties to this issue #754 https://github.com/SeleniumHQ/selenium-ide/issues/754#issuecomment-688308753

But the selenium-side-runner 3.17.0 is throwing this error: JavascriptError: javascript error: missing ) after argument list

I was not able to upgrade to v4 and i created a new task for it, becasue i am experiencing the same issue as in this thread. I do not want this to be fixed in 3.17.0, but you were saying that you have issues with the right click, so maybe it will get you on the right track.

[toddtarsi]

@karasma3 - Thanks for the tip. There's some good bits we could use here in v4. In v4, if we do support right click, we'd listen for that event to hook the recorder, but we'd only instrument it using the context click command in ide and side-runner. Using emulated playback is a bad road to go down, and a tool in the selenium ecosystem shouldn't instrument things that way. It's not that complex a code change, but I'm so buried in personal life that getting the new command and recorder out of me in the near term is physically impossible.

[ditorelo]

We had promises of this working mid-April last year - where we at team? 😬

[diemol]

Have you tried with the alpha releases of v4 before posting your complaint? :)

[toddtarsi]

@ditorelo - Yep! You should find v4 to be pretty good. The side runner, although in alpha, is up to like version 38 in npm (4.0.0-alpha.38). There are still some little things that are fiddly. Also, here is a link to the selenium-ide v4 releases. https://github.com/SeleniumHQ/selenium-ide/releases

The IDE is up to alpha version 32. You're right, I could probably do that as a beta, but for that I'd need code signing certificates, and that's a whole thing. For now, if the alpha of v4 isn't right for you, I totally get it. But its decent enough, in my highly biased opinion.

[FotisZal]

@toddtarsi Hi, issue is still seen in Selenium Side Runner. Tried in Windows 10 and 11. Many elements fail which can run successfully in IDE. Mainly consisting of react icons and buttons which open popups. Is there an update on this for Side Runner?

Xpath is for example: //div[@data-testid='place-of-performance']//button[@aria-label='Add icon']

and cmd logs:

element click intercepted: Element is not clickable at point (752, 998) (Session info: chrome=114.0.5735.91)

  at CommandNode.handleTransientError (node_modules/@seleniumhq/side-runtime/src/playback-tree/command-node.ts:191:17)
  at CommandNode.retryCommand (node_modules/@seleniumhq/side-runtime/src/playback-tree/command-node.ts:161:12)
  at CommandNode.execute (node_modules/@seleniumhq/side-runtime/src/playback-tree/command-node.ts:100:20)
  at Playback._executeCommand (node_modules/@seleniumhq/side-runtime/src/playback.ts:559:22)
  at Playback._executionLoop (node_modules/@seleniumhq/side-runtime/src/playback.ts:436:18)
  at Playback._executionLoop (node_modules/@seleniumhq/side-runtime/src/playback.ts:493:14)

console.error element click intercepted: Element ... is not clickable at point (752, 858). Other element would receive the click:

...

(Session info: chrome=114.0.5735.91)

  at CommandNode.handleTransientError (node_modules/@seleniumhq/side-runtime/src/playback-tree/command-node.ts:191:17)
  at CommandNode.retryCommand (node_modules/@seleniumhq/side-runtime/src/playback-tree/command-node.ts:161:12)
  at CommandNode.execute (node_modules/@seleniumhq/side-runtime/src/playback-tree/command-node.ts:100:20)
  at Playback._executeCommand (node_modules/@seleniumhq/side-runtime/src/playback.ts:559:22)
  at Playback._executionLoop (node_modules/@seleniumhq/side-runtime/src/playback.ts:436:18)
  at Playback._executionLoop (node_modules/@seleniumhq/side-runtime/src/playback.ts:493:14)

[supidupicoder2]

Also an issue on my side Tests work fine in IDE but when run with side runner i get Element ... is not clickable at point (x, y) errors

[toddtarsi]

v3 ide?

[toddtarsi]

Generally, everything here fails in this thread because v3 is shit. It uses emulated JS to pretend it's doing things, and then in the side-runner we're actually using the Selenium API correctly. The fix is to get on v4. It's one of the main reasons it exists.

v3 can't detect things like your button is actually covered so the user interaction doesnt work

[toddtarsi]

https://github.com/SeleniumHQ/selenium-ide/releases/tag/v4.0.1-beta.14