microsoft / playwright-pytest

Pytest plugin to write end-to-end browser tests with Playwright.
https://playwright.dev/python/docs/test-runners
Apache License 2.0
436 stars 69 forks source link

Make available artifacts path in test request #247

Open mnovait opened 1 week ago

mnovait commented 1 week ago

During the execution playwright produces different artifacts such as screenshots, videos or traces. Some of them are already save and made available in the class ArtifactsRecorder. I've created this PR to standardize how the plugin saves the paths as class attribute (videos were missing) and to copy in the test request the final result. The final goal is to have them in the test request, so they can be, for instance, used in the test report or somewhere else in the test flow.

Yes, but we usually start with an issue/feature-request, explain the use-case, elaborate on workarounds etc. Do you mind filing a feature request?

Originally posted by @mxschmitt in https://github.com/microsoft/playwright-pytest/issues/241#issuecomment-2388562443

mxschmitt commented 4 days ago

Thanks! I'd like to suggest an alternative: providing an output_dir fixture that returns the directory where test files are written. This would also let users write their own files to that location during the test execution (since its unique per test). This matches a request in https://github.com/microsoft/playwright-pytest/issues/227. Since you're already reading from the filesystem, using os.listdir would be needed.

Two questions for implementation:

  1. Which Pytest hook will you use for processing?
  2. Are you planning to upload these somewhere? Videos and tracing files only save on context fixture teardown, if you try to read/process before, you will get corrupted files.

Would this approach work for your needs?

mnovait commented 3 days ago

Thanks! I'd like to suggest an alternative: providing an output_dir fixture that returns the directory where test files are written. This would also let users write their own files to that location during the test execution (since its unique per test). This matches a request in #227. Since you're already reading from the filesystem, using os.listdir would be needed.

Two questions for implementation:

1. Which Pytest hook will you use for processing?

you can use pytest_runtest_logreport to run something like:

@pytest.hookimpl(tryfirst=True)
def pytest_runtest_logreport(report):
    if report.failed:
        log_files_message = (
            "it's a failure "
            f"screenshot {screenshot_path_here}"
        )
        if getattr(report.longrepr, "sections", None) is not None:
            report.longrepr.sections.append(
                (
                    "artifacts",
                    log_files_message,
                    "-",
                )
            )
2. Are you planning to upload these somewhere? Videos and tracing files only save on `context` fixture teardown, if you try to read/process before, you will get corrupted files.

what do you mean with upload? anyway, I'd try to access it only after the execution is ended... do you think that can work?

Would this approach work for your needs?