microsoft / playwright-dotnet

.NET version of the Playwright testing and automation library.
https://playwright.dev/dotnet/
MIT License
2.47k stars 235 forks source link

win32 exceptions when running on mac/linux #1183

Closed HaoK closed 3 years ago

HaoK commented 3 years ago

Hi, we are trying to use playwright sharp to do some of our E2E testing for Blazor on Asp.Net.Core, its working pretty great for Windows (see https://github.com/dotnet/aspnetcore/pull/30113), but we are running into some issues getting this to work on mac/linux. (https://github.com/dotnet/aspnetcore/pull/30404)

The overview of how our testing works right now is, our ci jobs build a single test payload via dotnet publish on windows, which we send to different test machines that run our tests, so windows, ubuntu, osx. I've tried a few different options to try and get playwright sharp working on mac/nix but so far none of these have worked:

Looking for some guidance for the best way to make this work.

Example Win32 exception error on ubuntu (is this because we ran the output of something published from windows?)

[xUnit.net 00:00:00.71]     Templates.Test.BlazorServerTemplateTest.BlazorServerTemplat_IdentityWeb_BuildAndPublish(auth: "IndividualB2C", args: ["--called-api-url \"https://graph.microsoft.com\"", "--called-api-scopes user.readwrite"]) [FAIL]
[xUnit.net 00:00:00.72]       System.ComponentModel.Win32Exception : Permission denied
[xUnit.net 00:00:00.72]       Stack Trace:
[xUnit.net 00:00:00.72]            at System.Diagnostics.Process.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec)
[xUnit.net 00:00:00.72]            at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
[xUnit.net 00:00:00.72]            at System.Diagnostics.Process.Start()
[xUnit.net 00:00:00.72]         /Users/neo/Documents/Coding/Personal/playwright-sharp/src/PlaywrightSharp/Transport/Connection.cs(51,0): at PlaywrightSharp.Transport.Connection..ctor(ILoggerFactory loggerFactory, TransportTaskScheduler scheduler, String driverExecutablePath, String browsersPath)
[xUnit.net 00:00:00.72]         /Users/neo/Documents/Coding/Personal/playwright-sharp/src/PlaywrightSharp/Playwright.cs(109,0): at PlaywrightSharp.Playwright.CreateAsync(ILoggerFactory loggerFactory, TransportTaskScheduler scheduler, String driverExecutablePath, String browsersPath, String debug)
[xUnit.net 00:00:00.72]         /_/src/Shared/BrowserTesting/src/BrowserManager.cs(49,0): at Microsoft.AspNetCore.BrowserTesting.BrowserManager.<InitializeAsync>g__InitializeCore|16_0()
[xUnit.net 00:00:00.72]         /_/src/Shared/BrowserTesting/src/BrowserManager.cs(45,0): at Microsoft.AspNetCore.BrowserTesting.BrowserManager.InitializeAsync()
[xUnit.net 00:00:00.72]         /_/src/Shared/BrowserTesting/src/BrowserManager.cs(38,0): at Microsoft.AspNetCore.BrowserTesting.BrowserManager.CreateAsync(IConfiguration configuration, ILoggerFactory loggerFactory)
[xUnit.net 00:00:00.72]         /_/src/ProjectTemplates/BlazorTemplates.Tests/PlaywrightFixture.cs(68,0): at ProjectTemplates.Tests.Infrastructure.PlaywrightFixture`1.InitializeAsync()
[xUnit.net 00:00:00.72]         /_/src/Testing/src/xunit/AspNetTestAssemblyRunner.cs(60,0): at Microsoft.AspNetCore.Testing.AspNetTestAssemblyRunner.<AfterTestAssemblyStartingAsync>b__2_0()
kblok commented 3 years ago

Hey @HaoK, First, the Win32Exception is an exception thrown by the Process class, no matter the platform :) Could you try v0.191.0? We changed the way we are deploying the driver. Maybe that might help.

HaoK commented 3 years ago

Cool I'm trying 191.1 now, I've verified that the drivers are getting delivered in the payload to the helix machines in the .playwright directory, and I'm hardcoding the paths to the root directory (that contains the .playwright),

I'm still seeing some errors, am I passing in the correct path for the driver path that you expect? C:\h\w\B70A09E6\w\9E8B096E\e\.playwright contains the drivers, previously it would throw a cannot find drivers exception, so this is progress!

Install code

                Console.WriteLine($"Installing Playwright to Browsers: {EnvironmentVariables["PLAYWRIGHT_BROWSERS_PATH"]} Drivers: {EnvironmentVariables["PLAYWRIGHT_DRIVER_PATH"]}");
                await Playwright.InstallAsync(EnvironmentVariables["PLAYWRIGHT_BROWSERS_PATH"]);                    await Playwright.InstallAsync(EnvironmentVariables["PLAYWRIGHT_BROWSERS_PATH"], EnvironmentVariables["PLAYWRIGHT_DRIVER_PATH"]);
Installing Playwright to Browsers: C:\h\w\B70A09E6\w\9E8B096E\e\ms-playwright Drivers: C:\h\w\B70A09E6\w\9E8B096E\e
Exception installing playwright: System.ComponentModel.Win32Exception (5): Access is denied.
   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at PlaywrightSharp.Transport.Connection.InstallAsync(String driverPath, String browsersPath) in /home/runner/work/playwright-sharp/playwright-sharp/src/PlaywrightSharp/Transport/Connection.cs:line 112
   at RunTests.TestRunner.InstallPlaywrightAsync() in C:\h\w\B70A09E6\w\9E8B096E\e\RunTests\TestRunner.cs:line 103

On linux/mac there's an error about FileName property

Installing Playwright to Browsers: /datadisks/disk1/work/AC810A03/w/A7550918/e/ms-playwright Drivers: /datadisks/disk1/work/AC810A03/w/A7550918/e
Exception installing playwright: System.ComponentModel.Win32Exception (0x80004005): The FileName property should not be a directory unless UseShellExecute is set.
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at PlaywrightSharp.Transport.Connection.InstallAsync(String driverPath, String browsersPath) in /home/runner/work/playwright-sharp/playwright-sharp/src/PlaywrightSharp/Transport/Connection.cs:line 112
   at RunTests.TestRunner.InstallPlaywrightAsync() in /datadisks/disk1/work/AC810A03/w/A7550918/e/RunTests/TestRunner.cs:line 103
pavelfeldman commented 3 years ago

I believe this has been resolved in https://www.nuget.org/packages/Microsoft.Playwright/1.11.1-alpha-2. Please file a new issue if this still happens.