quipucords / camayoc

Test automation framework that facilitates functional testing of quipucords.
https://camayoc.readthedocs.io/
GNU General Public License v3.0
5 stars 4 forks source link

UI scan tests - provide better error message in case of failure #459

Closed mirekdlugosz closed 11 months ago

mirekdlugosz commented 11 months ago

Sometimes UI tests fail when trying to download a scan report (usually because scan never started - thanks, thread-based scan manager!). Right now error message is rather unhelpful:

        credential_dto, source_dto, trigger_scan_dto = create_endtoend_dtos(source_name, data_provider)
        (
            ui_client.begin()
            .login(data_factories.LoginFormDTOFactory())
            .navigate_to(MainMenuPages.CREDENTIALS)
            .add_credential(credential_dto)
            .navigate_to(MainMenuPages.SOURCES)
            .add_source(source_dto)
            .trigger_scan(trigger_scan_dto)
            .navigate_to(MainMenuPages.SCANS)
            .download_scan(trigger_scan_dto.scan_form.scan_name)
            .logout()
        )

        is_network_scan = source_dto.select_source_type.source_type == SourceTypes.NETWORK_RANGE
        downloaded_report = ui_client.downloaded_files[-1]
        report_directory = Path(tempfile.mkdtemp(prefix="camayoc"))
>       report_file = report_directory / downloaded_report.suggested_filename
E       AttributeError: 'NoneType' object has no attribute 'suggested_filename'

With that patch applied, error message will be clearer:

        credential_dto, source_dto, trigger_scan_dto = create_endtoend_dtos(source_name, data_provider)
        (
            ui_client.begin()
            .login(data_factories.LoginFormDTOFactory())
            .navigate_to(MainMenuPages.CREDENTIALS)
            .add_credential(credential_dto)
            .navigate_to(MainMenuPages.SOURCES)
            .add_source(source_dto)
            .trigger_scan(trigger_scan_dto)
            .navigate_to(MainMenuPages.SCANS)
>           .download_scan(trigger_scan_dto.scan_form.scan_name)
            .logout()
        )

camayoc/tests/qpc/ui/test_endtoend.py:91: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
camayoc/ui/decorators.py:93: in inner
    page = func(*args, **kwargs)
camayoc/ui/decorators.py:25: in inner
    return func(*args, **kwargs)
camayoc/ui/decorators.py:47: in inner
    page = func(*args, **kwargs)
camayoc/ui/models/pages/scans.py:43: in download_scan
    downloaded_report = scan.download_scan()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <camayoc.ui.models.pages.scans.ScanListElem object at 0x7f3952136dd0>

    def download_scan(self) -> Download:
        scan_locator = "td.pf-c-table__action button[data-ouia-component-id=download]"
        timeout_start = time.monotonic()
        while 10 * 60 > (time.monotonic() - timeout_start):
            try:
                with self.locator.page.expect_download() as download_info:
                    self.locator.locator(scan_locator).click(timeout=10_000)
                download = download_info.value
                download.path()  # blocks the script while file is downloaded
                return download
            except PlaywrightTimeoutError:
                self._client.driver.locator(
                    "div.pf-c-toolbar button[data-ouia-component-id=refresh]"
                ).click()
>       raise FailedScanException("Scan could not be downloaded")
E       camayoc.exceptions.FailedScanException: Scan could not be downloaded
codecov[bot] commented 11 months ago

Codecov Report

Merging #459 (fe1023a) into main (31e9d97) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #459   +/-   ##
=======================================
  Coverage   77.07%   77.07%           
=======================================
  Files           4        4           
  Lines         205      205           
=======================================
  Hits          158      158           
  Misses         47       47           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more