Unable to set relative filepath to .srprofile within <Profile> of .runsettings

Konnor5092 commented 5 years ago

SpecFlow Version:

[x] 3.0
[ ] 2.4
[ ] 2.3
[ ] 2.2
[ ] 2.1
[ ] 2.0
[ ] 1.9

Used Test Runner

[x] SpecFlow+Runner
[ ] MSTest
[ ] NUnit
[ ] Xunit

Version number: 3.0.391

Project Format of the SpecFlow project

[x] Classic project format using packages.config
[ ] Classic project format using <PackageReference> tags
[ ] Sdk-style project format

.feature.cs files are generated using

[x] SpecFlow.Tools.MsBuild.Generation NuGet package
[ ] SpecFlowSingleFileGenerator custom tool

Visual Studio Version

[x] VS 2019
[ ] VS 2017
[ ] VS 2015

`Enable SpecFlowSingleFileGenerator Custom Tool` option in Visual Studio extension settings

[ ] Enabled
[ ] Disabled

Are the latest Visual Studio updates installed?

[x] Yes
[ ] No, I use Visual Studio version <Major>.<Minor>.<Patch>

.NET Framework:

[x] >= .NET 4.5
[ ] before .NET 4.5
[ ] .NET Core 2.0
[ ] .NET Core 2.1
[ ] .NET Core 2.2
[ ] .NET Core 3.0

Test Execution Method:

[x] Visual Studio Test Explorer
[ ] TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK
[ ] Command line – PLEASE SPECIFY THE FULL COMMAND LINE

<SpecFlow> Section in app.config or content of specflow.json

<specFlow>
  <stepAssemblies>
  <stepAssembly assembly="SpecFlow.Assist.Dynamic" />
    <stepAssembly assembly="Selenium.Framework" />
  </stepAssemblies>
</specFlow>

Repro Project

Issue Description

When attempting to build and discover tests for my solution, I get the following error in the Output -> Tests window

[07/10/2019 4:36:33.418 PM Informational] ---------- Discovery started ----------
[07/10/2019 4:36:35.578 PM Error] An exception occurred while test discoverer 'SpecRunTestDiscoverer' was loading tests. Exception: Could not find a part of the path 'C:\TFS\Smart End Point Assessment\Main\Selenium\Integration.SEPA\bin\Test\Settings\srprofile\Test\testmanual.srprofile'.

This was working prior to upgrading to 3.0.391 (I was already on version 3). Note I am selecting a .runsettings file prior to building.

I can see what's happening - It's taking the first part of the URL from the currently building project -> C:\TFS\Smart End Point Assessment\Main\Selenium\Integration.SEPA\bin\Test, and appending what I have in my .runsettings file...

  <SpecRun>
    <Profile>Settings\srprofile\Test\testmanual.srprofile</Profile>
    <GenerateSpecRunTrait>false</GenerateSpecRunTrait>
    <GenerateFeatureTrait>false</GenerateFeatureTrait>
  </SpecRun>

The project referenced by this error is not a specflow project. The .runsettings file resides in a different project which is where I would like the test discovery to look.

Setting an absolute path like below does resolve the issue, however this isn't viable when multiple people will be using the project and wont necessarily have their workspaces mapped the same

<Profile>C:\TFS\Smart End Point Assessment\Main\Selenium\Selenium.SEPA\bin\Test\Settings\srprofile\Test\testmanual.srprofile</Profile>

How do I a) define which project(s) to look in when performing the specflow+ test discovery during a rebuild of the solution?

or b) somehow ignore certain projects for test discovery whilst still being able to rebuild all projects in the solution at the same time.

Many thanks,

SabotageAndi commented 5 years ago

You can configure in the srProfile in which TestAssemblies the discovery and execution should be made. Docs is here: https://specflow.org/plus/documentation/TestAssemblyPaths/ Generally we look into all assemblies which references the TechTalk.SpecRun.dll.

Konnor5092 commented 5 years ago

@SabotageAndi The issue is finding the .srprofile via the .runsettings file, which presumably occurs before the .srprofile test assemblies are checked.

I already have that setting set...

  <TestAssemblyPaths>
    <TestAssemblyPath>Selenium.SEPA.dll</TestAssemblyPath>
  </TestAssemblyPaths>

And I can confirm this filename is only generated in one project.

Regarding the path referenced in the error...

C:\TFS\Smart End Point Assessment\Main\Selenium\Integration.SEPA\bin\Test\Settings\srprofile\Test\testmanual.srprofile

The Integration.SEPA project doesn't even have SpecFlow installed but it's still trying to find a .srprofile here when I build

Thanks,

rkkreddy commented 3 years ago

I see the same issue as well. @SabotageAndi, any update on this, when this will be fixed?

SabotageAndi commented 3 years ago

@rkkreddy No update on this issue.

SabotageAndi commented 2 years ago

As we stopped maintaining the SpecFlow+ Runner (https://specflow.org/using-specflow/the-retirement-of-specflow-runner/) I am closing this issue.

github-actions[bot] commented 2 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

SpecFlowOSS / SpecFlow