microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.4k stars 3.63k forks source link

[Bug]: /test-results subdirectories exceeds the default Windows pathname limit #29719

Closed chekrd closed 7 months ago

chekrd commented 7 months ago

Version

1.41.2

Steps to reproduce

Create tests with long test name (at least 100 characters) and expect a snapshot. Run Playwright in a directory with at least 60 characters long pathname.

Expected behavior

There is a way to customize the outputDir subdirectories name patter. Or PW lowers the 100 characters cap.

Actual behavior

PW creates a subdirectory for each test run. The directory name length seems to be capped to 100 + suffix. Another 100 for the screenshot itself. It means, that it is easy to reach 257 characters Windows filename limit: C: / some / directories / ... / test-results / 100 characters long test directory name / 100 characters long test file name

Additional context

Is there a way to customize the subdirectory name pattern? If not, can it be added? Something like this (example):

outputDir: string | ((testInfo: TestInfo) => string)

Environment

Windows
yury-s commented 7 months ago

@chekrd can you try to increase MAX_PATH as described in this document? We can probably trim to less than 100 characters but there is still a chance the same 260 chars limit will be exceeded if there are more directories.

chekrd commented 7 months ago

Increasing max_path can be a solution, but it requires investigation on our side how it will affect the whole system and it seems to be impossible to find all possible incompatibility issues, maybe this is the reason why it is not set on by default + it means that we must update the registry for all dev and infra machines. It seems to be very hard way in a context of running Playwright... Is there a chance to introduce something like this?

outputDir: string | (() => string) or adding a setting for max name length generated by PW?

chekrd commented 7 months ago

I tried another way around: I have created a custom reporter and I hoped I can read the testResult.body: Buffer and write the attachments in our own way (https://playwright.dev/docs/api/class-testresult) but the file was already created on disk with testResult.body set to undefined. Is there a way to disable implicit test-results write and force PW to give us the buffer? I consider this a hack, but I am still interested in how can I get the buffer 🙂

yury-s commented 7 months ago

Is there a way to disable implicit test-results write and force PW to give us the buffer?

No, the screenshot matcher writes the screenshots into the files right away, it doesn't send them around as buffers.