Initial IronPython compatibility

[GoogleCodeExporter]

We should check which kind of modifications are needed to Robot Framework 
to get full functionality under IronPython. It is already possible to run 
at least some test cases with IronPython, see below for details.

Separate issues should be opened for each specific modification, this 
issue just acts as remainder to keep this compatibility on radar.

To get Robot Framework running under IronPython one must:

1. Copy expatbuilder.py from under Python\Lib\xml\dom to IronPython\Lib\xml
\dom
2. Copy expat from under Python\Lib\xml\parsers to IronPython\Lib\xml
\parsers
3. download https://fepy.svn.sourceforge.net/svnroot/fepy/trunk/lib/
pyexpat.py and 
put it in the IRONPYTHONPATH (IronPython\Lib is in IRONPYTHONPATH always)
4. invoke Robot runner directly like `ipy C:\Python25\Lib\site-packages
\robot\runner.py`

Original issue reported on code.google.com by janne.t....@gmail.com on 20 Nov 2008 at 7:36

[GoogleCodeExporter]

IronPython 2.0 RC 2 has been released. Release notes mention that they are 
aiming to
get the final out in two weeks.

http://www.codeplex.com/IronPython/Wiki/View.aspx?title=v2.0%20Release%20Candida
te%202%20Release%20Notes

Original comment by pekka.klarck on 27 Nov 2008 at 4:59

[GoogleCodeExporter]

IronPython 2.0 has been released already some time ago. Janne, could you at some
point test how it works and report what's the procedure in taking it to use? 
Are the
steps in the original description still valid?

There's some interest in agile-testing list [1] about running RF on .NET so it 
would
be nice to have document the procedure on how to try it out. It might be a good 
idea
to create a separate wiki page for information related to this. 

[1] http://tech.groups.yahoo.com/group/agile-testing/message/16209

Original comment by pekka.klarck on 10 Feb 2009 at 6:05

[GoogleCodeExporter]

Just to give some feedback:
with the procedure above you get an error running the quickstart.html test.
WindowsError: (193, '[Errno 193] The specified executable is not a valid Win32
application.')

Robot Framework 2.0.4 (Python 2.6.1 on win32)
IronPython 2.0 (2.0.0.0) on .NET 2.0.50727.1433

Original comment by kris...@gmail.com on 31 Mar 2009 at 1:10

[GoogleCodeExporter]

The application used by quickstart.html only works with CPython. Can you run any
other tests?

Original comment by pekka.klarck on 31 Mar 2009 at 2:31

[GoogleCodeExporter]

Even for quickstart.html the logging generated and so was OK.
I've ran some other tests (examples) they were working fine.

Still have to look into it to make my own tests ...

Original comment by kris...@gmail.com on 2 Apr 2009 at 7:01

[GoogleCodeExporter]

I have to run Robot Framework 2.1.2 (RF) with IronPython 2.6 final. We have 
done 
test automation of WEB aplication succesfully with SeleniumLibrary 2.2.2. For 
WIN32 
we have used 'pyWinAuto'. I use HTML-Format with Winword because of the 
gigantic 
documentation and commentation posibilities. Much thanks for building such a 
good 
framework.
I must do more test automation with Windows-GUI's (WIN32, WPF, Windows Forms, 
etc.). 
But now under .NET we will/must switch to MS UI Automation (avaiable >= .NET 
3.0 NS: 
System.Window.Automation).

I have done the following efforts to get RF running with IronPython:

1. Installed IronPythton 2.6 succesfully

2. I have tryed to Install RF with robotframework-2.1.2.win32.exe (once more). 
It 
will always install to the (old) Python 2.6 directorys because of 
Registry-Values. 
Installation stopped.

3. Renamed the Reg-Key for Python (so it is no longer seen!). Install for RF 
will 
not let install to alternative free location. Install stopped because of no go.

4. Reg-Key for Python rerenamed and patched to point to Installation from 
IronPython. Next try to install with robotframework-2.1.2.win32.exe crashed!.

5. Manually copied the RF things below/and into the IronPython installation (to 
the 
same points as in python). All test automation is running, but at the end, 
preparing 
the HTML-Output from XML-Output failes.

6. I've done all advices from Janne in the Issue 154. In point 2. the file 
should 
read 'expat.py' instead of 'expat'. In point 4. I had no IRONPYTHONPATH. 
Therefore I 
copied 'pyexpat.py' to the lib dir and created a new IRONPYTHONPATH environment 
variable with this lib path.

Now the creation of the HTML-Output failes (in a other way) with following 
console 
output (ending):
--------------------------------------------------------------------------------
--
Output:  d:\eigene_dateien_wolfram\swtest\patools-test\winxp\tx_2009
\datentraeger\tests\01_contentsversions\results\output-20100127-150115.xml
[ ERROR ] Resolving variable '${suite.critical_stats.failed}' failed: 
IndexError: 
Der Index war außerhalb des Arraybereichs.

Try --help for usage information.
--------------------------------------------------------------------------------
-

The report output file was created but interruoted after the following contents:
--------------------------------------------------------------------------------
-
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 
Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="Expires" content="Mon, 20 Jan 2001 20:01:21 GMT" />
<meta name="generator" content="Robot 2.1.2 (Python 2.6.0 on cli)" />
<style media="all" type="text/css">
  /* Background color (green or red) */
  body {
--------------------------------------------------------------------------------

I think it's only a little bug because of the difference of the involved 
software 
pakages:
Robot Framework 2.0.4 => 2.1.2
IronPhython     2.6.1 => 2.6.10920.0 final on win32
.NET            2.0.50727.1433 => 2.0.50727.3603

I give You such a detailed report, because I think I'am not the one and only, 
who is 
interested in keyworddriven testautomation with RF and the newer MS UI 
Automation 
Layer to automate WIN applications regardless of the type of MS UI framework 
they 
use.

Original comment by wolfram....@googlemail.com on 27 Jan 2010 at 3:52

[GoogleCodeExporter]

Original comment by janne.t....@gmail.com on 24 Feb 2010 at 10:13

• Added labels: Target-2.5

[GoogleCodeExporter]

A part of IronPython support is changing the installation system so that the
framework can be installed without CPython. We also need new `ipybot` start-up 
script.

Original comment by pekka.klarck on 24 Feb 2010 at 6:37

[GoogleCodeExporter]

Now, after several days of fighting with RF against IPY 2.6.2 RC1 etc. I will 
publish my experiences.

As reported in comment 6, I patched an existing RF installation successfully 
beneath 
the IronPython(IPY) installation.
The reported bug on creating the report file could be arounded by commenting 
out the 
creation of the report file in RF code.
The following creation of the log file was succesful!.

The next was to copy an IPYBOT.BAT file from pybot.bat. This was easy because I 
only 
had to change the following two lines: 
  set python="C:\Programme\IronPython 2.6\ipy.exe"
  set runner="C:\Programme\IronPython 2.6\lib\site-packages\robot\runner.py"

Now I created a Testcase (WinWord-HTML) to start the 'application under test' 
(APT).

For the first test I created a IPY script with the following code:
(Examplecode from http://www.ironpython.info/index.php/Launching_Sub-Process)
-----------------------------------------------------
import time
from System.Diagnostics import Process

proc = Process()
proc.StartInfo.FileName = '<FilespecTo.exe>'
print 'Now starting'
proc.Start()
print 'Start done; Waiting for input idle'
proc.WaitForInputIdle()
print 'input is idle'

time.sleep(5)

print 'proc.id = %d' % proc.Id
print 'proc.mwh: %d' % proc.MainWindowHandle
-----------------------------------------------------

As a IPY-Script, this code does his job succesfully EVERY TIME IT IS CALLED.

Now I have implmented this code in a RF-Testlibrary as a keyword:
-----------------------------------------------------
import time
from System.Diagnostics import Process

class PAtestGuiAppLibrary:

    ROBOT_LIBRARY_SCOPE   = 'GLOBAL'

    def __init__(self, debug_flags = '0x0', std_wait = '0.0'):
    . . . .

    def start_application(self, azPath, azAppl, timeout = '1'):
        proc = Process()
        proc.StartInfo.FileName = '<azPath/azAppl>'
        print '*INFO* Now starting'
        proc.Start()
        print '*INFO* Start done; Waiting for input idle'
        proc.WaitForInputIdle()
        print '*INFO* input is idle'

        time.sleep(5)

        print '*INFO* proc.id = %d' % proc.Id
        print '*INFO* proc.mwh: %d' % proc.MainWindowHandle
        return proc.Id
-----------------------------------------------------

NOW HEAR THE VERY STRENGTH BEHAVIOR OF THIS CODE:
=================================================
1. The startup of the 'application under test' (APT) was successful
2. AFTER the line proc.Start() NOW MORE LINES WHERE DONE!!! All frozen!
3. The testcase runs into the testcase timeout (set to 1 min). Returning no 
proc.ID 
(= 0).
4. After the end of creating the output file and before creating the log file 
the 
remainding lines after proc.Start() where done!!!
5. Using the single line code 'Proccess.Start('<filespec>') has the same 
problem. 
After start the APT the line freezes.
6. I have tested all kinds of 'Run ...' keywords from OS library an got the 
same 
behaviour. They freeze after starting the 'aplication under test'. 
7. I have tested this under VisualStudio-Debugger and all breakpoints AFTER! 
the 
ProcStart-Line where not hit in time. They hit all to late near the end of RF 
programm. 
8. The probs are not specific to the APT, beausce starting a 'NOTEPAD.EXE' I 
get the 
same probs.

9. And now 'the Hammer': Today I found (accidentally due to ONE! COMPLETE 
SUCCESSFUL 
RUN after a nightly windows update w/ restart) that the above keyword 
implementation 
ONLY runs ONCE! after a fresh 'restart' of WINDOWS. This can be reproduced many 
times!

I am using WINDOWS XP Prof SP3 Installation running on a ESX-Virtual Machine.

Now, after a succesfully presentation from RF (with applause from the 
development 
team!), the things get very critical! 
Until today I have not yet reported this state to my boss. After that report, 
may 
be, I could no longer 'rise the flag for RF' and had to seek for an other 
keyword 
driven test framework.

I'am so frustated.
Sorry.

Wolfram

Original comment by wolfram....@googlemail.com on 19 Apr 2010 at 11:24

[GoogleCodeExporter]

Thanks for investigating this Wolfram. The problem you've encountered sounds 
really
strange, especially the fact that your code executes fine w/o RF but fails when 
run
as a library.

The only "special" thing the framework does before running keywords is 
intercepting
stdout and stderr. Perhaps that affects starting processes under IronPython 
somehow?
Could you test does disabling this functionality help? You can do that by 
replacing
`capture_output` and `replace_output` methods inside 
robot/utils/outputcapture.py
with these implementations:

def capture_output():
    pass

def release_output():
    return '', ''

Original comment by pekka.klarck on 19 Apr 2010 at 12:25

[GoogleCodeExporter]

There's no currently need for this inside NSN, thus this will not be included 
in RF 
2.5 unless we get external contribution.

Original comment by janne.t....@gmail.com on 5 May 2010 at 10:56

• Removed labels: Target-2.5

[GoogleCodeExporter]

I will be using IronPython and Robot Framework in our verification process. At 
this
time, my simple experiments following the original instructions from November 
2008
have worked to allow me to import an IronPython module, use its functions and 
test
results from them then report in Robot Framework.
It is probable that the extent of our use does not require functionality that 
causes
conflicts in IronPython. 
The basic environment is:
C# setup on the host system
CPython 2.6
IronPython 2.6
Robot Framework 2.1 on Win32( Haven't used it for our Linux applications yet)

The IronPython library calls into a C# executable for an xml log. 
I use the standard xml parsers available in both CPython and IronPython to test 
the log. 
So far, so good.

I am not sure what I will encounter as I develop more but I will report to you.
Thanks for the instructions on setting up and for following up on this issue.

Original comment by bugs...@gmail.com on 5 May 2010 at 11:20

[GoogleCodeExporter]

Hello all,

meanwhile I solved all my problems reported at comment 9 and manny others. Most 
problems are kind of blocking when using MS UI-Automation calls via IronPython 
and also beneath in custom written C# wrapper class lib.

The following forum contribution gives me an idea to test the complette 
environment on a REAL PC:
http://social.msdn.microsoft.com/forums/en-US/windowsaccessibilityandautomation/
thread/7f0bdc7c-be85-4fde-9f8a-cbb3f16ba5f4/
(see the contribution from E RaijarSekhar Reddy at Wednesday, May 05, 2010 8:00 
AM)
You will believe or not!
After switching my Test-PC (VMware ESXi Version 4) to my real Office-PC all my 
probs are gone!
Weeks of heavy developing workarounds like 
1) starting the application for testing seperately in the BAT-file in front of 
the start of the test (because Process.Start(...) never returns)
2) working around blocking Invoke-Call with seperate Thread (will not work! 
because then it blocks whenn calling the next UI-Automation method).
3) Implementing 'windows opened' event handler for aplication windows, (witch 
works badly because the run of the event code may be delayed until the end of 
the test (end of RF).

REMEMBER:
When you get (blocking) problems with MS UI Automation and you are on a Virtual 
Machine (VM), first leve the VM and switch to a REAL PC. Then you have a BIG 
chance to loose all problems!

Now I am sooooo happy!

Original comment by wolfram....@googlemail.com on 10 Aug 2010 at 1:45

[GoogleCodeExporter]

Hi,
 I have done the steps as per wolfram. But I'm getting the error like 'ImportError: No module named unicodedata' I hope, I'm getting this error due to java classes. 

How can I rectify this error? Please help me.

Thanks,
Palani.

Original comment by palani.s...@gmail.com on 8 Nov 2010 at 2:38

[GoogleCodeExporter]

This issue has been actively discussed on robotframework-users recently:
http://groups.google.com/group/robotframework-users/browse_thread/thread/6d87483
d0bb97286

Original comment by pekka.klarck on 11 Nov 2010 at 4:49

[GoogleCodeExporter]

I did some work to get Robot to run on IronPython 2.6.1 [1] based on the 
discussion on robotframework-users and committed the changes in revision 4281. 
After these changes, and before RF 2.5.5 with these fixes is released, the 
needed steps to run Robot on IronPython are:

1) Install IronPython 2.6.1 from [1] (or newer if available). 
2) Install elementtree module from [2]. Tested only with the 1.2.7 preview 
release, which should have better IronPython support than 1.2.6.
3) Checkout Robot Framework source code from [3] and go to the created 
directory from the command line.
4) Run tests like `ipy src\robot\runner.py tests.txt`.
5) Optionally you can install Robot like `ipy setup.py install`. Compiling 
source code into pyc files as part of the installation seems to fail for weird 
ValueError, but that doesn't affect the installation otherwise.
6) Optionally create `ipybot.bat` start-up script like Wolfram suggested in 
comment 9.

[1] http://ironpython.codeplex.com/releases/view/36280
[2] http://effbot.org/downloads/#elementtree
[3] http://code.google.com/p/robotframework/source/checkout

Original comment by pekka.klarck on 11 Nov 2010 at 5:03

[GoogleCodeExporter]

Workarounds for IronPython and ElementTree bugs with non-ASCII characters in 
r4288 and r4289.

I'm now able to run at least simple tests with IronPython with the setup 
described in the previous comment. I think that if we document these steps in a 
separate wiki page that's is already enough to say that IronPython is supported 
initially. Separate issues can be then submitted about generating a separate 
start-up script, creating libraries with C#, etc. Small 
enhancements/fixes/workarounds can and will still be included into RF 2.5.5.

Original comment by pekka.klarck on 17 Nov 2010 at 4:09

• Changed title: Initial IronPython compatibility
• Changed state: Started
• Added labels: Priority-High, Target-2.5.5
• Removed labels: Priority-Medium

[GoogleCodeExporter]

I run Robot's own acceptance tests with IronPython after adding few more 
workarounds in revision 4290. The end result was that 133 critical tests out of 
2085 failed. 133 sounds like a big number, but that's only 6.4% of the total 
and, more importantly, most of the failures are not very severe. In fact, a big 
number of test fails because IronPython represents Unicode strings a bit 
differently than CPython and Jython.

Original comment by pekka.klarck on 18 Nov 2010 at 3:43

[GoogleCodeExporter]

I've created DotNetSupport wiki page to describe the steps needed to get Robot 
installed on IronPython. Feedback is appreciated.

Original comment by pekka.klarck on 2 Dec 2010 at 12:17

[GoogleCodeExporter]

1) The current code base, and forthcoming 2.5.5, support IronPython pretty well.
2) The steps needed to use IronPython are described in the DotNetSupport wiki 
page.
3) We have separate issues about ipybot start-up script (issue 480) and C# 
support (issue 721).

=> We can consider initial IronPython compatibility to be done.

Original comment by pekka.klarck on 7 Dec 2010 at 12:53

• Changed state: Done

[GoogleCodeExporter]

I was wondering, based on this thread and the wiki page on .NET support, it's 
not made crystal clear but can I assume that:

There is (partial) support for RF normal operation and with IronPython. For 
example, Python (or ActivePython) installed + IronPython and .NET installed on 
Windows. Now you can install RF the normal way with/for regular Python via the 
Windows installer, then call/run RF with IronPython instead. And build test 
libraries using IronPython, even though RF was installed the normal way, not 
via the wiki method, though this requires you run RF through IronPython of 
course.

For simplicity let's assume regular Python and IronPython versions pretty much 
match up (v2.7).

If my assumptions are correct, would be nice to add that to the wiki page to 
clarify. I would think more simple to install RF the normal way and then use it 
with IronPython than to have to compile from sources with IronPython.

On a side note, is there any plans to make a Windows installer for RF that 
installs for IronPython rather than regular Python? That would be my wishlist 
item that I can submit if needed. So we don't need to compile RF sources.

Original comment by manga...@gmail.com on 19 Jan 2011 at 10:23

[GoogleCodeExporter]

Can RF be installed side by side, one install for IronPython, and another for 
Python/Jython?

Original comment by manga...@gmail.com on 19 Jan 2011 at 10:34