Full or relative path for baseline image directory

MininAS commented 3 years ago

Describe the bug What can we set like path for baseline image directory? Should it be full or relative path? With full path a test finishes correctly.

*** Settings ***
Documentation     Documentation

Library           SeleniumLibrary
Library           RobotEyes

Suite Setup       Suite Setup
Suite Teardown    Close All Browsers
Force tags        google_logo

*** Variables ***
${IMAGES_DIR}     ${OUTPUT_DIR}${/}visual_images_original

*** Test cases ***
Google Logo Should Not Be Changed
    Open Eyes      SeleniumLibrary    tolerance=5
    Wait Until Element Is Visible     css:.k1zIA.rSk4se
    Capture Element                   css:.k1zIA.rSk4se    name=image
    Compare Images
    [Teardown]    No Operation

*** Keywords ***
Suite Setup
    Open Browser    https://www.google.com/    chrome

Console output =>

But if we pointed relative path

*** Variables ***
${IMAGES_DIR}     visual_images_original

Console output =>

It is not blocker for testing localy, but it seems it affects to visual report at Jenkins automation server. We need to publish visual report results to artifactory and report contains broken link to base line image. An actual and diff links are correct. I assume that this is related that we set the baseline image directory path like full.

jessezach commented 3 years ago

@MininAS Relative Baseline path is supported. From the error log it looks like the failure occurs where it expects the output directory to exist. A condition above the assert statement checks for the existence of baseline folder. Seems like that's working fine. Is your baseline folder within the output folder? And are you passing absolute path to the output directory?

MininAS commented 3 years ago

Yep, baseline folder ${OUTPUT_DIR}${/}visual_images_original is in output folder. As far as the OUTPUT_DIR framework variable contains result output folder.

MininAS commented 3 years ago

And are you passing absolute path to the output directory? Yep. In this case OUTPUT_DIR = d:\robot-test-asset

jessezach commented 3 years ago

@MininAS Is this a separate location from where your tests are running?

MininAS commented 3 years ago

the same

report_generation.py =>

import os
import xml.etree.ElementTree as ET
from time import sleep

from .constants import IMAGE_EXTENSIONS

def generate_report(baseline_folder, results_folder, actual_folder=None):
    sleep(1)
    img_path = os.path.join(results_folder, 'visual_images')
    report_path = os.path.join(results_folder, 'output.xml')
    relative_baseline_folder_path = relative_path(baseline_folder, results_folder.replace(os.getcwd(), ''))
    relative_actual_folder_path = relative_path(actual_folder, results_folder.replace(os.getcwd(), '')) if actual_folder else ''

baseline_folder = "visual_images_original"
results_folder = "D:\robot-test-asset"

so => results_folder.replace(os.getcwd(), '') returns an empty because os.getcwd() = "D:\robot-test-asset" then relative_path method gets the empty the second attribute and go to exception

jessezach commented 3 years ago

@MininAS Understood. The code is breaking when results are stored in the root itself. I'll fix this

MininAS commented 3 years ago

Thank you!!! I'm looking forward to it. :)

jessezach commented 3 years ago

Hi @MininAS, I have pushed a fix in v1.5.7. Can you please upgrade and check if your issue gets resolved?

MininAS commented 3 years ago

Big thanks!!! Works good locally and in automation Jenkins too.

Wutsarawuthiveground commented 3 years ago

Hello @MininAS

      Could you please share the command to start your test?

so I had facing issue error >> 'please provide image baseline directory'

My command is >> robot -d regression -v image_dir:out1 .\robot-test..

MininAS commented 3 years ago

Hi! Your varible name is image_dir and should be images_dir.

jessezach / RobotEyes

Full or relative path for baseline image directory #88