Generating of report looks like broken.

MininAS commented 3 years ago

I don't know why it happenes only now but I get other issue related with generate report.

Console output:

Exception in thread Thread-33:
Traceback (most recent call last):
  File "C:\Program Files\Python36\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Program Files\Python36\lib\threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python36\lib\site-packages\RobotEyes\report_generator.py", line 68, in generate_report
    folder_name)
  File "C:\Program Files\Python36\lib\site-packages\RobotEyes\report_generator.py", line 227, in make_test_table
    for filename in os.listdir(baseline_folder + os.path.sep + folder_name):
FileNotFoundError: [WinError 3] The system cannot find the path specified: "visual_images_original\\Arguments: [ 'SeleniumLibrary' | tolerance=5 | template_id='1' ]"

A little invesigation: report_generator.py

jessezach commented 3 years ago

@MininAS Is your test a template test (data driven) one?

MininAS commented 3 years ago

Yep, you are right it is template. Sorry that I forgot to mention it

MininAS commented 3 years ago

Documentation     ...... 

Library           RobotEyes

Suite Setup       Suite Setup
Suite Teardown    Close All Browsers

*** Variables ***
${IMAGES_DIR}     visual_images_original

*** Test cases ***
Compare Some Image Rendering
    [Template]    Compare Some Images
    1    xxxxxxxxx
    2    yyyyyyyy
    3    zzzzzzzzz

*** Keywords ***
Suite Setup
    Open Browser    https://www.xxxxxx.com/    chrome

Compare Some Images
    [Arguments]    ${uid}    ${name}
    Open Eyes      SeleniumLibrary    tolerance=5    template_id=${uid}
    Open Some Page                    ${name}
    Wait Until Element Is Visible               css:some_selector    
    Capture Element                             css:some_selector    name=image
    Compare Images

jessezach commented 3 years ago

@MininAS ok, I will take a look. Did you by any chance edit the open_eyes function in the init.py file?

jessezach commented 3 years ago

Also could you provide the robot framework version you are on?

MininAS commented 3 years ago

No, didn't any editing.

Python 3.6.8 robotframework=3.1.2 robotframework-seleniumlibrary=4.3.0 robotframework-eyes=1.5.7 selenium=3.141.0

jessezach commented 3 years ago

Okay Thanks. I think the first element with msg tag is the list of Arguments passed to open_eyes. I think RF logs the Arguments by default. It's most likely a bug. I will fix this as soon as I get some time today or tomorrow. I'm planning to add tests so these type of issues can be avoided. However it is something that needs time. If you would like to help out by adding some regular Robot tests within a tests folder. Please feel free to do so

MininAS commented 3 years ago

I'll try to find time for that :)

jessezach commented 3 years ago

Hi @MininAS, I'm trying to reproduce this issue, however I'm unable to. Everything is working fine in my local machine using the same version of robotframework and selenium as yours. Here is what the open eyes keyword looks like in my output.xml

<kw name="Open Eyes" library="RobotEyes">
<arguments>
<arg>SeleniumLibrary</arg>
<arg>template_id=${uid}</arg>
</arguments>
<msg timestamp="20210410 12:22:31.811" level="INFO">Sample_test6_0</msg>
<status status="PASS" starttime="20210410 12:22:31.803" endtime="20210410 12:22:31.811"></status>
</kw>

As you can see there is only one element with msg tag. Can you share your output.xml with me, at least the open eyes keyword alone so I can see the xml generated for you and make the necessary fix.

MininAS commented 3 years ago

I use --loglevel TRACE:INFO

<kw name="Open Eyes" library="RobotEyes">
<arguments>
<arg>SeleniumLibrary</arg>
<arg>tolerance=5</arg>
<arg>template_id=${uid}</arg>
</arguments>
<msg timestamp="20210410 10:05:35.435" level="TRACE">Arguments: [ 'SeleniumLibrary' | tolerance=5 | template_id='1' ]</msg>
<msg timestamp="20210410 10:05:35.447" level="INFO">Compare_Bar_Chart_Rendering_1</msg>
<msg timestamp="20210410 10:05:35.447" level="TRACE">Return: None</msg>
<status status="PASS" starttime="20210410 10:05:35.435" endtime="20210410 10:05:35.447"/>
</kw>

jessezach commented 3 years ago

@MininAS Ah understood! That explains why there are multiple msg elements. Let me fix this. For the timebeing, if you remove loglevel, report generation should work fine

jessezach commented 3 years ago

@MininAS Released a fix in 1.5.8. Please upgrade. Let me know if it works

MininAS commented 3 years ago

Works fine with and without loglivel!!! Thanks a lot for fast support

jessezach / RobotEyes

Generating of report looks like broken. #90