search

microsoft / appcenter

Central repository for App Center open source resources and planning.
https://appcenter.ms
Creative Commons Attribution 4.0 International
1.01k stars 223 forks source link

Not able to run tests on app center using Xamarin.UITest version 4.2.0 - 4.3.2 #2562

Closed Moshex closed 1 year ago

Moshex commented 1 year ago

not able to run tests on app center using Xamarin.UITest version 4.2.0 but it works on 4.1.4.

After updating Xamarin.UITest to version 4.2.0-4.3.0 all of our pipelines broke and I needed to revert back to 4.1.4 to fix the issue. Below is an example of what happens in 4.2.0 and 4.1.4

This is 4.2.0 - 4.3.2:

{command} appcenter test run uitest --app "{app}" --devices 3664e0fd --app-path {app path} --test-series "Single Test" --locale "en_US" --build-dir {build directory}--include-category "SingleRun" --include "app.INI" --test-parameter "test_env=BundleId={app bundle}"
- Preparing tests... WARNING: Testing tests built for .NET 6 and above is not yet supported on Test Cloud.

/ Preparing tests... No tests would be run in Xamarin Test Cloud, reasons:

Preparing tests... failed.
Error: Failed to prepare UI Test artifacts using command "test-cloud.exe prepare" with error message:
No tests would run given the current parameters, please try again. If you can't work out how to fix this issue, please contact support.

Further error details: For help, please send both the reported error above and the following environment information to us by going to https://appcenter.ms/apps and opening a support ticket (using the help menu in the upper right corner of the screen)

    Environment: darwin
    App Upload Id: {id}
    Timestamp: 1689695499908
    Operation: RunUITestsCommand
    Exit Code: 21
    User Email: {user email}
    User Name: {user name}
    User Id: {user id}

This is 4.1.4

{command} appcenter test run uitest --app "{app}" --devices 3664e0fd --app-path {app path} --test-series "Single Test" --locale "en_US" --build-dir {build directory}--include-category "SingleRun" --include "app.INI" --test-parameter "test_env=BundleId={app bundle}"
\ Preparing tests... WARNING: Testing tests built for .NET 6 and above is not yet supported on Test Cloud.

Preparing tests... done.
Validating arguments... done.
Creating new test run... done.
Validating application file... done.
Uploading files... done.
Starting test run... done.
Test run id: "2b5aad32-6f2f-4d41-a04a-9c4669262be6"
Accepted devices: 
  - Samsung Galaxy S23+  (13)

Current test status: Validation completed. Waiting for devices

Current test status: Validation completed. Waiting for devices

Current test status: Validation completed. Waiting for devices

Current test status: Validation completed. Waiting for devices

Current test status: Validation completed. Waiting for devices

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Running on 1 device (0 / 1 completed, 0 pending)

Current test status: Tests completed. Processing data.

Current test status: Tests completed. Processing data.

Current test status: Tests completed. Processing data.

Current test status: Done!
Total scenarios: 1
1 passed
0 failed
Total steps: 1

Test Report: {test report link}
### Tasks
gustavoja commented 1 year ago

Hello! same issue here

IlyaBausovAkvelon commented 1 year ago

@Moshex Hi there! Thank you for reporting it. Could you please provide link to your run?

Moshex commented 1 year ago

@IlyaBausovAkvelon On what version? with 4.2.0 there is no link since it fails to run the test at all, however, on 4.1.4 it just works as expected, so what link would you be looking for?

IlyaBausovAkvelon commented 1 year ago

@Moshex could you please provide dependencies' versions that you're using in tests VS project?

Moshex commented 1 year ago

@IlyaBausovAkvelon I submitted a support ticket about the app center issue and I was told that internally you are aware of the issue, Is that the case? and if YES any ETA on the next release?

DmitriyKirakosyan commented 1 year ago

Hi @Moshex , Could you please share the .csproj file from your test project? We'd like to review the dependencies and their versions.

Also, would it be possible for you to share a demo test project for us to reproduce the issue on it?

Thank you for your patience!

Moshex commented 1 year ago

@DmitriyKirakosyan I have added you as a contributor to a demo project that I have made so you can grab the project and take a look for yourself. I will also post below my .csproj file text without personal project information.

<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Import Project="..\packages\Xamarin.UITest.4.2.0\build\Xamarin.UITest.props" Condition="Exists('..\packages\Xamarin.UITest.4.2.0\build\Xamarin.UITest.props')" />
  <Import Project="..\packages\Xamarin.UITest.4.1.4\build\Xamarin.UITest.props" Condition="Exists('..\packages\Xamarin.UITest.4.1.4\build\Xamarin.UITest.props')" />
  <Import Project="..\packages\SpecFlow.NUnit.3.9.74\build\SpecFlow.NUnit.props" Condition="Exists('..\packages\SpecFlow.NUnit.3.9.74\build\SpecFlow.NUnit.props')" />
  <Import Project="..\packages\SpecFlow.Tools.MsBuild.Generation.3.9.74\build\SpecFlow.Tools.MsBuild.Generation.props" Condition="Exists('..\packages\SpecFlow.Tools.MsBuild.Generation.3.9.74\build\SpecFlow.Tools.MsBuild.Generation.props')" />
  <Import Project="..\packages\NUnit.3.13.3\build\NUnit.props" Condition="Exists('..\packages\NUnit.3.13.3\build\NUnit.props')" />
  <Import Project="..\packages\SpecFlow.Plus.LivingDocPlugin.3.9.57\build\SpecFlow.Plus.LivingDocPlugin.props" Condition="Exists('..\packages\SpecFlow.Plus.LivingDocPlugin.3.9.57\build\SpecFlow.Plus.LivingDocPlugin.props')" />
  <Import Project="..\packages\Xamarin.Forms.5.0.0.2012\build\Xamarin.Forms.props" Condition="Exists('..\packages\Xamarin.Forms.5.0.0.2012\build\Xamarin.Forms.props')" />
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <OutputType>Library</OutputType>
    <RootNamespace>WellRx.UITests</RootNamespace>
    <AssemblyName>WellRx.UITests</AssemblyName>
    <TargetFrameworkVersion>v4.8</TargetFrameworkVersion>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
    <DebugSymbols>true</DebugSymbols>
    <DebugType>full</DebugType>
    <Optimize>false</Optimize>
    <OutputPath>bin\Debug</OutputPath>
    <DefineConstants>DEBUG;</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <CustomCommands>
      <CustomCommands>
        <Command>
          <type>BeforeBuild</type>
          <command></command>
          <workingdir>${ProjectDir}</workingdir>
        </Command>
      </CustomCommands>
    </CustomCommands>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
    <Optimize>true</Optimize>
    <OutputPath>bin\Release</OutputPath>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'AppStore|AnyCPU' ">
    <Optimize>true</Optimize>
    <OutputPath>bin\AppStore</OutputPath>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'AppStore|iPhone' ">
    <Optimize>false</Optimize>
    <OutputPath>bin\iPhone\AppStore</OutputPath>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'QA|AnyCPU' ">
    <Optimize>true</Optimize>
    <OutputPath>bin\QA</OutputPath>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'QA|iPhone' ">
    <Optimize>false</Optimize>
    <OutputPath>bin\iPhone\QA</OutputPath>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'CI|AnyCPU' ">
    <Optimize>true</Optimize>
    <OutputPath>bin\CI</OutputPath>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'CI|iPhone' ">
    <Optimize>false</Optimize>
    <OutputPath>bin\iPhone\CI</OutputPath>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>
  <ItemGroup>
    <Reference Include="System" />
    <Reference Include="Microsoft.CSharp" />
    <Reference Include="mscorlib" />
    <Reference Include="Mono.Android">
      <HintPath>..\..\..\..\..\Library\Frameworks\Mono.framework\External\xbuild-frameworks\MonoAndroid\v8.1\Mono.Android.dll</HintPath>
    </Reference>
    <Reference Include="nunit.framework">
      <HintPath>..\packages\NUnit.3.13.3\lib\net45\nunit.framework.dll</HintPath>
    </Reference>
    <Reference Include="TechTalk.SpecFlow">
      <HintPath>..\packages\SpecFlow.3.9.74\lib\net461\TechTalk.SpecFlow.dll</HintPath>
    </Reference>
    <Reference Include="Newtonsoft.Json">
      <HintPath>..\packages\Newtonsoft.Json.13.0.2\lib\net45\Newtonsoft.Json.dll</HintPath>
    </Reference>
    <Reference Include="Xamarin.Forms.Core">
      <HintPath>..\packages\Xamarin.Forms.5.0.0.2012\lib\netstandard2.0\Xamarin.Forms.Core.dll</HintPath>
    </Reference>
    <Reference Include="Xamarin.Forms.Platform">
      <HintPath>..\packages\Xamarin.Forms.5.0.0.2012\lib\netstandard2.0\Xamarin.Forms.Platform.dll</HintPath>
    </Reference>
    <Reference Include="Xamarin.Forms.Xaml">
      <HintPath>..\packages\Xamarin.Forms.5.0.0.2012\lib\netstandard2.0\Xamarin.Forms.Xaml.dll</HintPath>
    </Reference>
    <Reference Include="System.Security.Cryptography.Encoding">
      <HintPath>..\packages\System.Security.Cryptography.Encoding.4.3.0\lib\net46\System.Security.Cryptography.Encoding.dll</HintPath>
    </Reference>
    <Reference Include="System.Security.Cryptography.Primitives">
      <HintPath>..\packages\System.Security.Cryptography.Primitives.4.3.0\lib\net46\System.Security.Cryptography.Primitives.dll</HintPath>
    </Reference>
    <Reference Include="System.Core" />
    <Reference Include="System.Security.Cryptography.X509Certificates">
      <HintPath>..\packages\System.Security.Cryptography.X509Certificates.4.3.2\lib\net461\System.Security.Cryptography.X509Certificates.dll</HintPath>
    </Reference>
    <Reference Include="System.Net.Http">
      <HintPath>..\packages\System.Net.Http.4.3.4\lib\net46\System.Net.Http.dll</HintPath>
    </Reference>
    <Reference Include="SendGrid">
      <HintPath>..\packages\SendGrid.9.28.1\lib\net452\SendGrid.dll</HintPath>
    </Reference>
    <Reference Include="System.ValueTuple">
      <HintPath>..\packages\System.ValueTuple.4.5.0\lib\net47\System.ValueTuple.dll</HintPath>
    </Reference>
    <Reference Include="System.IO">
      <HintPath>..\packages\System.IO.4.3.0\lib\net462\System.IO.dll</HintPath>
    </Reference>
    <Reference Include="System.Runtime">
      <HintPath>..\packages\System.Runtime.4.3.1\lib\net462\System.Runtime.dll</HintPath>
    </Reference>
    <Reference Include="System.ComponentModel.Composition" />
    <Reference Include="System.Security.Cryptography.Algorithms">
      <HintPath>..\packages\System.Security.Cryptography.Algorithms.4.3.1\lib\net463\System.Security.Cryptography.Algorithms.dll</HintPath>
    </Reference>
    <Reference Include="BoDi">
      <HintPath>..\packages\BoDi.1.5.0\lib\net45\BoDi.dll</HintPath>
    </Reference>
    <Reference Include="Google.Protobuf">
      <HintPath>..\packages\Google.Protobuf.3.7.0\lib\net45\Google.Protobuf.dll</HintPath>
    </Reference>
    <Reference Include="Cucumber.Messages">
      <HintPath>..\packages\Cucumber.Messages.6.0.1\lib\net45\Cucumber.Messages.dll</HintPath>
    </Reference>
    <Reference Include="System.Threading.Tasks.Extensions">
      <HintPath>..\packages\System.Threading.Tasks.Extensions.4.5.4\lib\net461\System.Threading.Tasks.Extensions.dll</HintPath>
    </Reference>
    <Reference Include="Utf8Json">
      <HintPath>..\packages\Utf8Json.1.3.7\lib\net47\Utf8Json.dll</HintPath>
    </Reference>
    <Reference Include="Gherkin">
      <HintPath>..\packages\Gherkin.19.0.3\lib\net45\Gherkin.dll</HintPath>
    </Reference>
    <Reference Include="TechTalk.SpecFlow.NUnit.SpecFlowPlugin">
      <HintPath>..\packages\SpecFlow.NUnit.3.9.74\lib\net461\TechTalk.SpecFlow.NUnit.SpecFlowPlugin.dll</HintPath>
    </Reference>
    <Reference Include="LivingDoc.Dtos">
      <HintPath>..\packages\SpecFlow.Plus.LivingDocPlugin.3.9.57\lib\net461\LivingDoc.Dtos.dll</HintPath>
    </Reference>
    <Reference Include="LivingDoc.SpecFlowPlugin">
      <HintPath>..\packages\SpecFlow.Plus.LivingDocPlugin.3.9.57\lib\net461\LivingDoc.SpecFlowPlugin.dll</HintPath>
    </Reference>
    <Reference Include="SpecFlow.Internal.Json">
      <HintPath>..\packages\SpecFlow.Internal.Json.1.0.8\lib\net45\SpecFlow.Internal.Json.dll</HintPath>
    </Reference>
    <Reference Include="StarkbankEcdsa">
      <HintPath>..\packages\starkbank-ecdsa.1.3.3\lib\net452\StarkbankEcdsa.dll</HintPath>
    </Reference>
    <Reference Include="System.Runtime.Serialization" />
    <Reference Include="System.Runtime.CompilerServices.Unsafe">
      <HintPath>..\packages\System.Runtime.CompilerServices.Unsafe.6.0.0\lib\net461\System.Runtime.CompilerServices.Unsafe.dll</HintPath>
    </Reference>
    <Reference Include="System.Configuration" />
    <Reference Include="System.Security.Principal.Windows">
      <HintPath>..\packages\System.Security.Principal.Windows.5.0.0\lib\net461\System.Security.Principal.Windows.dll</HintPath>
    </Reference>
    <Reference Include="System.Security.AccessControl">
      <HintPath>..\packages\System.Security.AccessControl.6.0.0\lib\net461\System.Security.AccessControl.dll</HintPath>
    </Reference>
    <Reference Include="System.Security.Permissions">
      <HintPath>..\packages\System.Security.Permissions.7.0.0\lib\net462\System.Security.Permissions.dll</HintPath>
    </Reference>
    <Reference Include="System.Data.OracleClient" />
    <Reference Include="System.Net" />
    <Reference Include="System.Security" />
    <Reference Include="System.ServiceProcess" />
    <Reference Include="System.Transactions" />
    <Reference Include="WindowsBase" />
    <Reference Include="System.Configuration.ConfigurationManager">
      <HintPath>..\packages\System.Configuration.ConfigurationManager.7.0.0\lib\net462\System.Configuration.ConfigurationManager.dll</HintPath>
    </Reference>
    <Reference Include="System.Configuration.Install" />
    <Reference Include="System.Xml" />
    <Reference Include="System.Net.Http.Formatting">
      <HintPath>..\packages\Microsoft.AspNet.WebApi.Client.5.2.9\lib\net45\System.Net.Http.Formatting.dll</HintPath>
    </Reference>
    <Reference Include="ICSharpCode.SharpZipLib">
      <HintPath>..\packages\SharpZipLib.1.4.2\lib\netstandard2.0\ICSharpCode.SharpZipLib.dll</HintPath>
    </Reference>
    <Reference Include="System.Security.Cryptography" />
    <Reference Include="System.Buffers">
      <HintPath>..\packages\System.Buffers.4.5.1\lib\net461\System.Buffers.dll</HintPath>
    </Reference>
    <Reference Include="System.Numerics.Vectors">
      <HintPath>..\packages\System.Numerics.Vectors.4.5.0\lib\net46\System.Numerics.Vectors.dll</HintPath>
    </Reference>
    <Reference Include="System.Numerics" />
    <Reference Include="System.Memory">
      <HintPath>..\packages\System.Memory.4.5.5\lib\net461\System.Memory.dll</HintPath>
    </Reference>
    <Reference Include="Mono.Cecil">
      <HintPath>..\packages\Mono.Cecil.0.11.4\lib\net40\Mono.Cecil.dll</HintPath>
    </Reference>
    <Reference Include="Mono.Cecil.Mdb">
      <HintPath>..\packages\Mono.Cecil.0.11.4\lib\net40\Mono.Cecil.Mdb.dll</HintPath>
    </Reference>
    <Reference Include="Mono.Cecil.Pdb">
      <HintPath>..\packages\Mono.Cecil.0.11.4\lib\net40\Mono.Cecil.Pdb.dll</HintPath>
    </Reference>
    <Reference Include="Mono.Cecil.Rocks">
      <HintPath>..\packages\Mono.Cecil.0.11.4\lib\net40\Mono.Cecil.Rocks.dll</HintPath>
    </Reference>
    <Reference Include="DocoptNet">
      <HintPath>..\packages\docopt.net.0.8.1\lib\netstandard2.0\DocoptNet.dll</HintPath>
    </Reference>
    <Reference Include="Xamarin.UITest">
      <HintPath>..\packages\Xamarin.UITest.4.2.0\lib\net462\Xamarin.UITest.dll</HintPath>
    </Reference>
    <Reference Include="Xamarin.UITest.Repl">
      <HintPath>..\packages\Xamarin.UITest.4.2.0\lib\net462\Xamarin.UITest.Repl.dll</HintPath>
    </Reference>
    <Reference Include="Xamarin.UITest.Shared">
      <HintPath>..\packages\Xamarin.UITest.4.2.0\lib\net462\Xamarin.UITest.Shared.dll</HintPath>
    </Reference>
  </ItemGroup>
  <ItemGroup>
    <Compile Include="TestSetups.cs" />
    <Compile Include="AppInitializer.cs" />
    <Compile Include="Base\By.cs" />
    <Compile Include="Base\Element.cs" />
    <Compile Include="Global.cs" />
    <Compile Include="Pages\Page.cs" />
    <Compile Include="Pages\WelcomePage.cs" />
    <Compile Include="Features\WelcomePage.feature.cs">
      <DependentUpon>WelcomePage.feature</DependentUpon>
    </Compile>
    <Compile Include="Steps\NavigationSteps.cs" />
    <Compile Include="Helper\IniFile.cs" />
  <Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
  <Import Project="..\packages\Xamarin.Forms.5.0.0.2012\build\Xamarin.Forms.targets" Condition="Exists('..\packages\Xamarin.Forms.5.0.0.2012\build\Xamarin.Forms.targets')" />
  <Import Project="..\packages\SpecFlow.Plus.LivingDocPlugin.3.9.57\build\SpecFlow.Plus.LivingDocPlugin.targets" Condition="Exists('..\packages\SpecFlow.Plus.LivingDocPlugin.3.9.57\build\SpecFlow.Plus.LivingDocPlugin.targets')" />
  <Import Project="..\packages\SpecFlow.3.9.74\build\SpecFlow.targets" Condition="Exists('..\packages\SpecFlow.3.9.74\build\SpecFlow.targets')" />
  <Import Project="..\packages\SpecFlow.Tools.MsBuild.Generation.3.9.74\build\SpecFlow.Tools.MsBuild.Generation.targets" Condition="Exists('..\packages\SpecFlow.Tools.MsBuild.Generation.3.9.74\build\SpecFlow.Tools.MsBuild.Generation.targets')" />
  <Import Project="..\packages\SpecFlow.NUnit.3.9.74\build\SpecFlow.NUnit.targets" Condition="Exists('..\packages\SpecFlow.NUnit.3.9.74\build\SpecFlow.NUnit.targets')" />
  <Import Project="..\packages\docopt.net.0.8.1\build\netstandard2.0\docopt.net.targets" Condition="Exists('..\packages\docopt.net.0.8.1\build\netstandard2.0\docopt.net.targets')" />
</Project>
Hackmodford commented 1 year ago

I'm having the same issue.

SimonBrettschneider commented 1 year ago

I am facing the same issue.

DmitriyKirakosyan commented 1 year ago

@Moshex , can I ask you to add @IlyaBausovAkvelon to your demo project as a contributor too? I can't share it with him without your approval.

Moshex commented 1 year ago

@DmitriyKirakosyan I have added @IlyaBausovAkvelon as a collaborator to the project, he should have the invite in his inbox.

Moshex commented 1 year ago

@DmitriyKirakosyan and @IlyaBausovAkvelon Any updates?

Moshex commented 1 year ago

@DmitriyKirakosyan and @IlyaBausovAkvelon I have updated to version 4.3.0 and now I have the following error:

`appcenter test run uitest --app "ScriptSave/DEV-WellRx-1" --devices 46286509 --app-path {app-path} --test-series "Single Test" --locale "en_US" --build-dir {build-dir} --include-category "SingleRun" --include "app.INI" --test-parameter "{test-parameter }"
- Preparing tests... WARNING: Testing tests built for .NET 6 and above is not yet supported on Test Cloud.

| Preparing tests... No tests would be run in Xamarin Test Cloud, reasons:

Preparing tests... failed.
Error: Failed to prepare UI Test artifacts using command "test-cloud.exe prepare" with error message:
No tests would run given the current parameters, please try again. If you can't work out how to fix this issue, please contact support.

Further error details: For help, please send both the reported error above and the following environment information to us by going to https://appcenter.ms/apps and opening a support ticket (using the help menu in the upper right corner of the screen)

    Environment: {Environment}
    App Upload Id: {App Upload Id}
    Timestamp: 1696877858224
    Operation: RunUITestsCommand
    Exit Code: 21
    User Email: {email}
    User Name: {user name}
    User Id: {user id}

is there anything I need to do to fix the issue?

dude3133 commented 1 year ago

Xamarin.UITest 4.3.1 release 2 days ago. Somebody tested if that fixed anything?

upd: as for running iOS device locally - nope

Moshex commented 1 year ago

@dude3133 so far on version Xamarin.UITest 4.3.1 I have not been able to get app center to work but I am still looking into it since the patch notes of Xamarin.UITest 4.3.1 claims to directly solve the issue.

Moshex commented 1 year ago

I am still seeing the same issue on Xamarin.UITest version 4.3.1 @DmitriyKirakosyan and @IlyaBausovAkvelon Any updates as to why?

Moshex commented 1 year ago

I am still seeing the same kind of issue on Xamarin.UITest version 4.3.2 @DmitriyKirakosyan and @IlyaBausovAkvelon Any updates as to why? here is the new error I am seeeing on the version 4.3.2:

appcenter test run uitest --app "ScriptSave/DEV-WellRx-1" --devices 46286509 --app-path /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.Droid/bin/Debug/com.wellrx.xf.apk --test-series "Single Test" --locale "en_US" --build-dir /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/ --include-category "SingleRun" --include "app.INI" --test-parameter "test_env=BundleId=com.wellrx.xf"
(node:6316) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
- Preparing tests... WARNING: Testing tests built for .NET 6 and above is not yet supported on Test Cloud.

| Preparing tests... Grouping tests.

Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/Mono.Cecil.Rocks.dll
Getting tests.

/ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/System.IO.dll
Getting tests.

\ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/System.Security.Permissions.dll
Getting tests.

/ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/Utf8Json.dll
Getting tests.

- Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/WellRx.UITests.dll
Getting tests.

\ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/Google.Protobuf.dll
Getting tests.

/ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/System.Net.Http.dll
Getting tests.

- Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/Cucumber.Messages.dll
Getting tests.

| Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/Gherkin.dll
Getting tests.

- Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/System.Security.Principal.Windows.dll
Getting tests.

| Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/System.Security.Cryptography.X509Certificates.dll
Getting tests.

- Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/SpecFlow.Internal.Json.dll
Getting tests.

\ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/Xamarin.Forms.Core.dll
Getting tests.

/ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/StarkbankEcdsa.dll
Getting tests.

\ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/SendGrid.dll
Getting tests.

/ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/System.Threading.Tasks.Extensions.dll
Getting tests.

- Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/Mono.Cecil.dll
Getting tests.

| Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/nunit.framework.dll
Getting tests.

- Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/ICSharpCode.SharpZipLib.dll
Getting tests.

| Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/System.Security.Cryptography.Primitives.dll
Getting tests.

/ Preparing tests... Analyzing assembly: /Users/mmoadeb/Documents/WellRx_MobileXF/WellRx.UITests/bin/Debug/Mono.Android.dll
Getting tests.

- Preparing tests... Test chunking failed: Unsupported Target Framework: MonoAndroid,Version=v1.0

Preparing tests... failed.
Error: Failed to prepare UI Test artifacts using command "test-cloud.exe prepare" with error message:
Test chunking failed. Please try to delete dlls which are not necessary from the path provided in the --build-dir argument.If this error happens again, please contact support.

Further error details: For help, please send both the reported error above and the following environment information to us by going to https://appcenter.ms/apps and opening a support ticket (using the help menu in the upper right corner of the screen)

    Environment: darwin
    App Upload Id: {id}
    Timestamp: 1701293664250
    Operation: RunUITestsCommand
    Exit Code: 13
    User Email: {user email}
    User Name: {user name}
    User Id: {user id}
IlyaBausovAkvelon commented 1 year ago

Hi @Moshex, What NUnitTestAdapter version do you use?

IlyaBausovAkvelon commented 1 year ago

Also why do you need Mono.Android.dll?

IlyaBausovAkvelon commented 1 year ago

This is happening because test-cloud.exe is trying to load test package from Mono.Android.dll. Here is a line with this exception in nunit-console repository: https://github.com/nunit/nunit-console/blob/945090ca56ddf36869c87b84b67f88b0cc0974d3/src/NUnitEngine/nunit.engine/Services/RuntimeFrameworkService.cs#L173

Moshex commented 1 year ago

@IlyaBausovAkvelon

I am on Nunit version 3.13.3 do I need to update my Nunit?

also, you were correct that deleting Mono.Android.dll allowed my tests to work on app center. However, Mono.Android.dll is made by the Xamarin.Android NuGet package which is a part of my overall project since my automation project lives with the project that it's automating as most Xamarin.UITest projects do, it seems to be generated in the automation bin folder. Now why Mono.Android.dll is being made in my Xamarin.UITest project bin folder I am not sure, but the bigger question is why app center is taking a dll that is breaking the run execution as a result. is there no why to do a quick update to account for the Mono.Android.dll?

Info on Mono.Android.dll:

Mono.Android.dll is a library that is part of the Xamarin.Android framework, which is a cross-platform mobile app development platform that allows developers to build apps for Android devices using C# and .NET. The library provides a set of APIs for interacting with the Android operating system and device hardware, as well as tools for building, debugging, and deploying Android apps. Mono.Android.dll is specifically responsible for providing the runtime environment that allows Xamarin.Android apps to run on Android devices. It is developed and maintained by Microsoft as part of the Xamarin framework.

The Mono.Android.dll is generated by the Xamarin.Android NuGet package. This package provides the necessary libraries and tools for developing and building Android apps with Xamarin. It includes the Mono runtime environment, the Xamarin.Android libraries, and various tools for building, debugging, and deploying Android apps. When you create a Xamarin.Android project in Visual Studio or another IDE, this package is typically installed automatically as a dependency. You can also install it manually using the NuGet package manager.

IlyaBausovAkvelon commented 1 year ago

@Moshex, good to hear that tests are working now. Does the workaround by removing this dll from artifacts work for you? If yes I will close this issue and will provide fix for it in the next release.

Moshex commented 1 year ago

it does and works in the pipeline, thank you @IlyaBausovAkvelon for all your help and the fix for this issue you have been amazing!

IlyaBausovAkvelon commented 12 months ago

@Moshex thank you!

Naveen1547 commented 10 months ago

Issue still persists image

Naveen1547 commented 10 months ago

is anyone have solution for this in MAUI

Moshex commented 10 months ago

@Naveen1547 If not mistaken this is just based on what APK or IPA you have. the fact that it's a MAUI app should not matter. What error are you getting?

IlyaBausovAkvelon commented 10 months ago

Hi @Naveen1547, As I see from the logs you're trying to upload tests built for .NET 8. No version of .NET is supported on Appcenter. Please try to build your test project for .NET Framework (your main app project can remain in .NET or Xamarin target framework).