In Selenium Server extension, loops not working

[apodhajsky]

hello, i have a problem : i'm trying to run tests writen in SeleniumIDE heavily dependend on SelBlocks (btw briliant work). But when i try to run them they do only 1 loop.

Firefox version: 33.0 Selenium version: 2.9.0 SelBlocks version: 2.1

cmd: java -jar selenium-server-standalone-2.45.0.jar -port 5555 -firefoxProfileTemplate "full_path_to_template" -singlewindow -debug -log ./htmlsuit.log -htmlSuite "*firefox" "https://address.sk" "/full_path_to_suite/sel_block_test.html" "result.html" -user-extensions "/full_path_to/user-extensions.js"

result:

info: [selblocks] Applying testCase server patch for selblocks info: Starting test /selenium-server/tests/for_loop_test info: Executing: |for | i=0; i<=10; i++ | | info: Executing: |getEval | LOG.info("${i}"); | | info: script is: LOG.info("0"); info: 0 info: Executing: |endFor | | | info: [selblocks] branch => @1: [for|i=0; i<=10; i++] info: Starting test /selenium-server/tests/while_test info: Executing: |store | 10 | k | info: Executing: |while | ${k} >= 0 | | info: Executing: |storeEval | "${k}"-1; | k | info: script is: "10"-1; info: Executing: |getEval | LOG.info("${k}"); | | info: script is: LOG.info("9"); info: 9 info: Executing: |endWhile | | | info: [selblocks] branch => @2: [while|10 >= 0]

as you can see, after 1st loop test finish

[apodhajsky]

it seems that no command is executed ater endLoop command selenium standalone server : 2.45.0 Firefox version: 33.0 Selenium version: 2.9.0 SelBlocks version: 2.1

for_loop_test
getEval	LOG.info("------------------------------------------------------------------------------");
store	0	checkVar
for	i=0; i<=10; i++
getEval	LOG.info(${i});
storeEval	(${i});	checkVar
endFor
getEval	LOG.info(${checkVar});
assertEval	${checkVar}	10
getEval	LOG.info("------------------------------------ END ------------------------------------------");

while_test
getEval	LOG.info("------------------------------------ BEGIN ------------------------------------------");
store	10	k
while	${k} >= 0
storeEval	${k}-1;	k
getEval	LOG.info("${k}");
endWhile
getEval	LOG.info("${k}");
assertEval	${k} -1
getEval	LOG.info("------------------------------------ END ------------------------------------------");

result info: [selblocks] Applying testCase server patch for selblocks info: Starting test /selenium-server/tests/for_loop_test info: Executing: |getEval | LOG.info("------------------------------------------------------------------------------"); | | info: script is: LOG.info("------------------------------------------------------------------------------"); info: ------------------------------------------------------------------------------ info: Executing: |store | 0 | checkVar | info: Executing: |for | i=0; i<=10; i++ | | info: Executing: |getEval | LOG.info(${i}); | | info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP info: script is: LOG.info(0); info: 0 info: Executing: |storeEval | (${i}); | checkVar | info: script is: (0); info: Executing: |endFor | | | info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP info: [selblocks] 2

info: [selblocks] branch => @3: [for|i=0; i<=10; i++] info: Executing: |for | i=0; i<=10; i++ | | info: Executing: |getEval | LOG.info(0); | | info: script is: LOG.info(0); info: 0 info: Executing: |storeEval | (0); | checkVar | info: script is: (0); info: Starting test /selenium-server/tests/while_test info: Executing: |getEval | LOG.info("------------------------------------ BEGIN ------------------------------------------"); | | info: script is: LOG.info("------------------------------------ BEGIN ------------------------------------------"); info: ------------------------------------ BEGIN ------------------------------------------ info: Executing: |store | 10 | k | info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP info: Executing: |while | ${k} >= 0 | | info: Executing: |storeEval | ${k}-1; | k | info: script is: 10-1; info: Executing: |getEval | LOG.info("${k}"); | | info: script is: LOG.info("9"); info: 9 info: Executing: |endWhile | | | info: [selblocks] 2

info: [selblocks] branch => @3: [while|10 >= 0] info: Executing: |while | 10 >= 0 | | info: Executing: |storeEval | 10-1; | k | info: script is: 10-1; info: Executing: |getEval | LOG.info("9"); | | info: script is: LOG.info("9"); info: 9

[refactoror]

@apodhajsky I just don't have time to try to reproduce this. Please note that the SelBlocks regression test suite covers this scenario, and it works correctly. Perhaps you could run the test suite in your environment and compare the results.

See: https://github.com/refactoror/SelBlocks/tree/master/sel-blocksTests The html result files for each browser are also in that directory.

[apodhajsky]

@refactoror i run regression tests and : 1.) on selenium IDE - all green 2.) on seleniumServer -htmlsuite - all red after removing log and emmit commands (becouse they are not part of user-extension.js) test dies after 1st cycle (in case of for loops) again just to be clear, there is error is when i run seleniumServer -htmlsuite in IDE everything works more than well

[refactoror]

SORRY. I forgot to mention that the test suite requires the SelBench plugin, as well. It provides emit, assert, etc. Without those commands, test success is not meaningful.

Get SelBench here: https://github.com/refactoror/SelBench/blob/master/user-extensions.js Concatenate this to the SelBlocks user-extensions.js, and run again.

By the way, what version of Firefox are you using? The goal is to support the latest ESR version of Firefox, (currently 31) - definitely not the monthly releases. https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/latest-esr/

One thing I'm concerned about is that Mozilla is planning some major changes to how addons are constrained, some of which may be starting to show up in the monthly releases. See: https://groups.google.com/forum/#!topic/selenium-users/p24nzz_3p-E I hope this is not related to that...

[apodhajsky]

sorry for late response i change job recently so less time to spend on this. btw. i think that browser version is irrelevant becouse selenium server is execution engine and not Selenium IDE running in browser. but i have Firefox 37.0.1 and i test it also on 31.6.0 i put selblocks and selbench togeter and yes, report is all green, but report Lies!!! i cut for.html test from regression testssuite and change cycle to go from 0 to 10 and as you can see at the bottom, it run only ONCE! even emit END is not in log

Test suite results result: passed totalTime: 0 numTestTotal: 2 numTestPasses: 2 numTestFailures: 0 numCommandPasses: 0 numCommandFailures: 0 numCommandErrors: 0 Selenium Version: 2.45 Selenium Revision: .0 Test Suite README for README.html README clearLog
open http://refactoror.net/_SelBlocks-testpage.html
for.html for resetEmitted deleteVars f
emit "START ${f}"
for s=0,f=0; f <= 10; f++
emit "iter=${f}.${s}"
endFor
emit "END ${f}" assertEmitted "START $" + "{f}~iter=3.0~iter=4.0~iter=5.0~END $" + "{f}"

---

info: [selbench] Applying testCase server patch for selbench info: Starting test /selenium-server/tests/README.html info: Executing: |clearLog | | | info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP warn: [selbench] clearLog command ignored in non-IDE environment info: Executing: |open | http://refactoror.net/_SelBlocks-testpage.html | |

---

info: Starting test /selenium-server/tests/for.html info: Executing: |resetEmitted | | | info: Executing: |deleteVars | f | | info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP info: Executing: |emit | "START ${f}" | | ****** 1st cycle begin **** info: Executing: |for | s=0,f=0; f <= 10; f++ | | info: Executing: |emit | "iter=${f}.${s}" | | info: Executing: |endFor | | | **** 1st cycle end **** info: [selblocks] branch => @4: [for|s=0,f=0; f <= 10; f++] **** 2nd cycle begin **** info: Executing: |for | s=0,f=0; f <= 10; f++ | | info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP info: Executing: |emit | "iter=0.0" | |**

[apodhajsky]

as far as i remember by debugging this behaviour it silently dies at command responsibe for execution of html line with for parameters recomputation and almost same behaviour is when i try emulate loops using if - goto construction btw. i also compare results and are same: this is from results for firefox from repository: info: Starting test /selenium-server/tests/for.html info: Executing: |resetEmitted | | | info: Executing: |deleteVars | f | | info: Executing: |emit | "START ${f}" | | info: Executing: |for | s=0,f=3; f <= 5; f++ | | info: Executing: |emit | "iter=${f}.${s}" | | info: Executing: |endFor | | | info: [selblocks] branch => @4: [for|s=0,f=3; f <= 5; f++] info: Executing: |for | s=0,f=3; f <= 5; f++ | | info: Executing: |emit | "iter=3.0" | | info: Starting test /selenium-server/tests/foreach.html

[refactoror]

Confirmed. Per the original bug description, "no command is executed after endLoop command". The reason that failed tests are reporting as successful is that the assertEmitted is never reached.

This is a huge hole in the testsuite, and every test needs to be re-scrutinized.

@apodhajsky, thanks for your persistence in reporting this.

[apodhajsky]

I need to be persistent, becouse not every tester is also java programmer able to wrile full webdriver test. But almost everybody have some basic programming skills needed to craft solid SeleniumIDE test with SelBlocks. And be able to run html suite on Selenium Grid is final goal that can help lot of people out there...

[medimohammadise]

I have the same problem. After 1'th iteration test case does not run anything.Do you have any update on this bug? info: [selblocks] branch => @2: [forJson|http://127.0.0.1:7101/INTLFunctionalTestSuite-TestDriverUI-context-root/test2015-05-27_09-08-16.json]

[medimohammadise]

Please Help me

[apodhajsky]

Hi, what you can do right now is to transform your SeleniumIDE (Selblocks) test into JUnit or TestNG test and run them (after few changes) from Eclipse or as jar file...

[refactoror]

Correct. That has been the assumption since the inception of Selblocks. Running directly via the server extension is experimental - more so than I had originally thought. However, we will continue to look in to getting it to work...

[medimohammadise]

I wonder that If I want to convert it to JUnit. How can I use Selblocks's related functionality such as forJson ?

[middreks]

I've bumped into the same issue while trying to run my test suites with Jenkins. Seems to have the same problem both with forXml and loadTestData. Right now we don't have the resources and capacity to transform our tests to JUnit. So could it be worth it to temper with the extension's code to try to make this work?

[refactoror]

@oraclemohammadi: forJson and forXml are higher features, so there is no equivalent in other languages. You would have to do something custom in those cases.

@middreks: By all means, tamper. This is open source, and participation is quite welcome. The server extension is merely the concatenation of a small amount of "shim" code, followed by the Selblocks code from the Firefox addon. See: build/createSelblocksUserExtensions.cmd

[medimohammadise]

@refactoror How can I contribute on fixing this issue ? (sel-blocks problem on webdriver server)

[refactoror]

See: https://github.com/refactoror/SelBlocks/tree/master/build/createSelblocksUserExtensions.cmd for assembling the user-extensions.js from the source.

To submit changes to source files other than user-extensions-base.js, (ie, the uderlying Firefox addon), fork the github project, and then submit pull requests. The trick is to change the addon code as little as possible. Note that the addon has a complete regression test suite.

[bobiben]

I have the same problem. loop while for ==> it didn't work for Selenium Core Please Help me !!!!

[apodhajsky]

just to make it clear: in selenium htmlSuite with selBlocks right now does not work any kind of loop - for, while, also goto loop. to regression - are they fixed? becouse results were not correct (green instead of red)

[refactoror]

Note that the regression tests were originally for the Firefox extension, (which does not have this problem). But they should be equally suitable for the server extension. The problem is that the tests are showing green because this bug causes tests to stop prematurely, ie, without reaching the assertEmitted, which would otherwise detect the shortfall in iterations.

I think the best way to validate results for the server extension now will be to first run the test suite in the Firefox extension, save that test log, and then use that as a comparison for the server extension results.

By the way, I did not write the user-extensions-base.js code that makes the extension work, but I have an unsubstantiated hunch that this might be related to the way comments are skipped over.

[bobiben]

I confirm the commande ( if , endIf) don't work too for Selenium Core

[refactoror]

ATTENTION - For users other that the original reporter, please go back and edit your comments to include the Name and Version of the browser(s) that you are running via Selenium Server.

[bobiben]

I use "selenium-server-standalone-2.45.0.jar" ,"seleniumIDE 2.9.0", "firefox 18.0.2" Because I've been trying to access the RESTClient extension in firefox using Selenium IDEI with the command " open | chrome://restclient/content/restclient.html withe firefox version firefox 37. I keep getting this error returned. [error] Unexpected Exception: Error: Access to 'chrome://restclient/content/restclient.html' from script denied. But it works with firefox 18.0.2 and firefox 14. !!

[medimohammadise]

I use : selenium-server-standalone-2.45.0.jar FireFox 37 Selenium IDE 2.9.0 Selblocks 2.1

[middreks]

I've been using Firefox 36.0.1 as I checked this it updated to 38.0.5 and now crashes... Trying to downgrade now if that helps. Tried with both server 2.45.0 and 2.46.0 with latest Selblocks. Developed on IDE 2.9.0 but until now was running on the server without any IDE parts installed.

EDIT: Downgrading to Firefox 31.7.0esr seems to have solved the problem. Unfortunately I couldn't save the crash log from the Interactive Services screen (and cannot find the logs anywhere either) but had to do something with openqa readystate. Maybe they changed something in FF38. Turned automatic updates off to be safe.

[medimohammadise]

What do you mean by "without any IDE parts installed"? you should use user-Eextenstions switch and inject userExtensions.js for running selBlocks on server.it is necessary off course.

[middreks]

I've been running the node with the following command:

java -jar selenium-server-standalone-2.46.0.jar -role node -port 5577 -hub http://localhost:4444/grid/register (changed url to localhost for the sake of this post) In Jenkins I use the following parameters: -log log.txt -userExtensions ${userext} -port 4455 -timeout 1200 (i have user-extensions.js as a global variable here)

I have been using it, but I didn't install the IDE or any plugins to the browser itself, that's what I meant. :)

Btw I will try to spend some time with figuring out the code how loops are made. Seems like there is a part which doesn't let the script to jump back to a specific command/label.

Here is a part from the test log:

info: Executing: |endWhile | | | info: [selblocks] branch => @3: [while|!testdata.EOF()] info: Starting test /selenium-server/tests/SecondTestCase.html

Branch @3 correctly states that it should go back to the while command to check for the condition again, but then it just jumps to the next test case.

[medimohammadise]

As I get ,you mean that you can successfully run the loops with Firefox 36.0 ? If you confirm this we can downgrade our Firefox to 36.0. Thanks

[middreks]

It wasn't working with FF36 either :) I had to downgrade to 31.7 to make sure the tests are running as most were crashing with FF38.

[medimohammadise]

We have already tested it with FireFox 31 . It would not work.

[middreks]

Just to clarify: I believe it doesn't work with any FF versions. I mentioned the downgrade because the browser itself crashed so all the tests (loop or not failed) but now are sliding away from the original issue :)

[refactoror]

Thanks. The browser version can be important, because the extension uses a relatively new javascript mechanism (Object.defineProperties) that may not be supported in all browsers.

[patrick-hubert-adsk]

I have a potential one-line fix that works at least for me when using the standalone Selenium server. I thought I might share with you guys (or anyone searching for a fix).

I modified my version of user-extensions.js as follow, in function nextCommand:

if (branchIdx !== null) {
    $$.LOG.info("branch => " + fmtCmdRef(branchIdx));
    this.debugIndex = branchIdx;
    testCase.htmlTestCase.nextCommandRowIndex = branchIdx;
    branchIdx = null;
}
else {

From what I could tell, the issue is with a discrepancy between the different instruction pointers (the SelBlocks one in debugIndex and the htmlTextCase one).

I am working on a Mac, which makes it difficult for me to run the createSelblocksUserExtensions.cmd and run-testsuites.cmd (as these are Windows CMD + cygwin based).

I cannot state that my code change will also work in the context of the IDE, or even in your own run environment... But it does work for me.

Hoping this helps someone.

[middreks]

Still didn't fix for me but it's progress :) I've made a simple test suite that searches for 3 different things in Google: one test case using forXml and one for loadTestData. Both run 3 times, but always load just the first record. The unmodified user-extenions.js still works fine in IDE so I won't bother trying it there. For long term it could be a good idea to put check there if its running in IDE or server.

[tn801534]

I test another write user-extensions.js test at selenium-server-standalone-2.53.0.jar for loop is ok https://github.com/matthewkastor/SelBlocks/blob/kastor-dev/user%20extension/user-extensions.js