SpecFlowOSS / SpecFlow

#1 .NET BDD Framework. SpecFlow automates your testing & works with your existing code. Find Bugs before they happen. Behavior Driven Development helps developers, testers, and business representatives to get a better understanding of their collaboration
https://www.specflow.org/
Other
2.23k stars 752 forks source link

Azure Pipeline on Linux - run same Project Multiple times in same Job gives Generation Error not able to find **/*.feature files on the 2nd Task #2599

Open chrisusher-instanda opened 2 years ago

chrisusher-instanda commented 2 years ago

SpecFlow Version

3.9.74

Which test runner are you using?

NUnit

Test Runner Version Number

3.13.2

.NET Implementation

.NET 5.0

Project Format of the SpecFlow project

Sdk-style project format

.feature.cs files are generated using

SpecFlow.Tools.MsBuild.Generation NuGet package

Test Execution Method

TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK

SpecFlow Section in app.config or content of specflow.json

{ "language": { "feature": "en-gb" }, "stepAssemblies": [ ], "livingDocGenerator": { "enabled": true, "filePath": "TestExecution.json" } }

Issue Description

This issue is not happening all the time but has started happening more and more often in our Azure Pipelines.

We have a Specflow Project we use multiple times in the same Job, just run with different --filter parameters

Pipeline YAML

I.e.

The first Task always runs fine, but then the second Task fails with :

Error Log

See attached Error log as Markdown messes with the file search patterns 😦

SpecflowGeneratorIssue.log

Summary

The first Task was able to find the Feature Files and generate .feature.cs files successfully and Run Tests but the 2nd Time it doesn't work.

Steps to Reproduce

Define an Azure YAML Pipeline

  1. Runs on ubuntu-latest
  2. Has a Job which has two DotNetCoreCLI@2 Tasks using the Same Specflow Project (see issue description)
  3. Test Project is Net6.0

Link to Repro Project

No response

chrisusher-instanda commented 2 years ago

SpecflowGeneratorIssue.log

SabotageAndi commented 2 years ago

You could try to add --no-build to the second task so that no build is triggered for it.

chrisusher-instanda commented 2 years ago

@SabotageAndi Yes, we could, but wondering why multiple runs on the same job doesn't work without it. The feature files are there, as are the .feature.cs files already.

I can run tests many time on a dev machine with no complaints about not being able to find the feature files from the MSBuild Generation task.

robertcoltheart commented 1 year ago

I'm getting this same issue running a build inside a docker container on Bamboo. I'm using 4.0.16-beta

**/*.feature : error : System.IO.DirectoryNotFoundException: Could not find a part of the path '/store/bamboo/agent-home/xml-data/build-dir/JOB/src/Tests/**/*.feature'. [/store/bamboo/agent-home/xml-data/build-dir/JOB/src/Tests/Tests.csproj]
**/*.feature : error :    at TechTalk.SpecFlow.Generator.FeatureFileInputExtensions.GetFeatureFileContentReader(FeatureFileInput featureFileInput, ProjectSettings projectSettings) [/store/bamboo/agent-home/xml-data/build-dir/JOB/src/Tests/Tests.csproj]
AvallaSD commented 11 months ago

Same problem on MacOS. Does anyone found a solution?

GaTechThomas commented 8 months ago

I'm seeing similar errors that show deeper stack trace that make calls to Windows libraries (Microsoft.Win32.SafeHandle.Open), but it's running on linux.