Open lanfeust69 opened 2 years ago
Why do you put multiple test assemblies in the same folder?
Why do you put multiple test assemblies in the same folder?
As I said :
to avoid wasting disk space, and also because there are runtime dependencies (plugins) that are not explicit dependencies of all test projects.
And anyway that should be the decision of the developer (or team), not forced upon by the tools they use, especially when it previously used to work (so that's a regression).
If we are using the word regression
here, it is that .NET Core doesn't support app.config
anymore. This is not a limitation we did.
We added the possibility to use specflow.json
for configuration with SpecFlow 2.2.0 (https://github.com/SpecFlowOSS/SpecFlow/blob/master/changelog.txt#L381) about 5 years ago. A lot earlier than SpecFlow got .NET Core support (which happens with 3.0).
You can use it in .NET Framework and .NET Core.
I see that this feature is missing, but you are also the first one in 5 years that would like to have it.
But here is the nice thing. SpecFlow is open-source and depends on contributors. I am always happy to review Pull Requests and help with implementing feature requests.
In your case, the code you are looking for is here: https://github.com/SpecFlowOSS/SpecFlow/blob/master/TechTalk.SpecFlow/Configuration/SpecFlowJsonLocator.cs All other configuration related code can be found here: https://github.com/SpecFlowOSS/SpecFlow/tree/master/TechTalk.SpecFlow/Configuration Unit tests are here: https://github.com/SpecFlowOSS/SpecFlow/blob/master/Tests/TechTalk.SpecFlow.RuntimeTests/Configuration/JsonConfigTests.cs Integration Tests are here: https://github.com/SpecFlowOSS/SpecFlow/blob/master/Tests/TechTalk.SpecFlow.Specs/Features/Execution/Configuration.feature
If you want to discuss how to implement it, we can do this here or on Discord (https://discord.com/invite/xQMrjDXx7a)
Indeed, I had taken a look at the code (to convince myself that I had not overlooked a way to specify the configuration file).
I will possibly give it a shot if I find a bit of time, but it is usually better to discuss the topic around an open issue than to come up directly with a PR (at least for non-trivial matters).
By the way, .NET Core does somewhat support app.config
, but I fully agree that the additional dependency to the corresponding package is something we'd much rather avoid.
Hello I have the same probleme, we have multiple specflow projects and each one with its specific configuration :( How did you solve that problem?
Hello @gabidabet,
Not solved. But for now we don't run .NET Core tests, so we can still live with the app.config
files. We also need configuration for the live-doc generation, which in practice needs to go in specflow.json
, but we only have a couple of projects that use it, so we can do it sequentially, overwriting the specflow.json
file between each project. Obviously, this won't scale, though...
Hello @gabidabet,
Not solved. But for now we don't run .NET Core tests, so we can still live with the
app.config
files. We also need configuration for the live-doc generation, which in practice needs to go inspecflow.json
, but we only have a couple of projects that use it, so we can do it sequentially, overwriting thespecflow.json
file between each project. Obviously, this won't scale, though...
Thank you for replying.
We would like to use the json configuration, even if our projects are .NET Framework. The reason to have separate json files is because even if locally the code is built into separate directories, the CI server builds it into a shared directory, which is then packaged into the deployment archive to be deployed on the various AT environments and ran there.
I am not a SpecFlow user, my interest lies in the deployment process. But it seems to me the code uses Dependency Injection and so the ISpecFlowJsonLocator
interface is injected into the ConfigurationLoader
constructor - https://github.com/SpecFlowOSS/SpecFlow/blob/ddab6a3e9c347f32b2dfb7fb89539c737c7a2225/TechTalk.SpecFlow/Configuration/ConfigurationLoader.cs#L22-L27
If I am correct, can we hook into the Dependency Injection configuration process and provide our own implementation of that interface?
@MarkKharitonov yeah, it uses dependency injection, but it will be resolved earlier than you are able to change it via a plugin.
@SabotageAndi - so the DI configuration does not have any hooks? Can you point me at this code? I am not a SpecFlowTest user, but I have a skin in the game when it comes to build, packaging and deployment.
@MarkKharitonov the DI has hooks, but they are all called after the ISpecFlowJsonLocator is already resolved.
@SabotageAndi
What are your thoughts on adding a file look up for "{assemblyname}.specflow.json" as a priority over "specflow.json". Possibly an additional option in the VS extension to "Prioritize assembly specific specflow.json configuration". Does the Locator have visibility to the extension input? If not that, what about build actions for file rename and copy to output directory? Could you point to code where build actions are implemented (i.e. where steps classes are generated)?
Thanks~
This is not the only location where the file is read. Also, the Visual Studio Extensions (<= VS2019, >= VS2022) and Rider is reading the file. The extensions are not connected to SpecFlow in any way.
But this is only relevant when you are in the IDE, not when you are running scenarios. So for this use case, it doesn't need to be adjusted. So, I think an additional if
in https://github.com/SpecFlowOSS/SpecFlow/blob/master/TechTalk.SpecFlow/Configuration/SpecFlowJsonLocator.cs#L10 could be enough.
The problem is the order of these if. I can't remember which of these cases are the generation part and runtime parts.
The copying of the specflow.json
to the output directory is done here: https://github.com/SpecFlowOSS/SpecFlow/blob/master/TechTalk.SpecFlow/build/SpecFlow.targets#L16
Entry point of code behind file generation is this MSBuild Task: https://github.com/SpecFlowOSS/SpecFlow/blob/master/SpecFlow.Tools.MsBuild.Generation/GenerateFeatureFileCodeBehindTask.cs
I had an idea in the past how to solve this issue, which as far as I remember covered runtime and generation time as well. I'll try to find my branch when I get home after work.
@lanfeust69 Could you please give some details about which configuration settings you would like to specify in the specflow.json
so that we can find some other suggestions?
BTW: I think SpecFlow is still able to read the config from the app.config
as well, so an easy solution for you would be to use the app.config
instead, I think...
@gasparnagy
@lanfeust69 Could you please give some details about which configuration settings you would like to specify in the
specflow.json
so that we can find some other suggestions?
I have actually moved from the project where specflow is used, but as I hinted in the initial description, the most crucial setting was specifying the assemblies where the steps are defined.
BTW: I think SpecFlow is still able to read the config from the
app.config
as well, so an easy solution for you would be to use theapp.config
instead, I think...
Not on .NET Core...
Greetings @gasparnagy!
Could you please give some details about which configuration settings you would like to specify in the
specflow.json
so that we can find some other suggestions?
TL;DR: I have Assembly C referencing Assembly B, and Assembly B referencing Assembly A:
C -> B -> A
I have a similar problem. In my repository we have one project Calculator using SpecFlow steps from another NuGet package. To point SF to that assembly we're using specflow.json
(number '1' on the screenshot).
But then I have another project/assembly SpecFlowTests where I need steps from Calculator (number '2' on the screenshot). Here I have a project reference to Calculator, as well as specflow.json
.
And if I build SpecFlowTests, I'll get only 1 specflow.json
in the output directory.
One idea which I'm thinking about is to use a single specflow.json
having all the settings, and referencing all the assemblies that I need. But it does look like a hack to me.
Maybe you could provide some suggestions?
@EvgenyMosk I have moved over to Reqnroll but this behavior (merging multiple config files) is currently not supported in either tools.
Thanks a lot for your reply, @gasparnagy!
SpecFlow Version
3.9.40
Which test runner are you using?
NUnit
Test Runner Version Number
3.13.1
.NET Implementation
.NET Core 3.1
Project Format of the SpecFlow project
Sdk-style project format
.feature.cs files are generated using
SpecFlow.Tools.MsBuild.Generation NuGet package
Test Execution Method
Command line
nunit3-console.exe SomeAssembly.dll ; nunit3-console.exe SomeOtherAssembly.dll
SpecFlow Section in app.config or content of specflow.json
Issue Description
We have all our test projects share the same ouput directory (to avoid wasting disk space, and also because there are runtime dependencies (plugins) that are not explicit dependencies of all test projects).
This is fine with the
app.config
approach, where different test projects can have different configurations (typically to specify step assemblies), but doesn't work with a single hard-codedspecflow.json
file (even if each test project had their own, they would end up overwriting each other during build). Since this is the only option for .NET Core, we have a regression there.Maybe an option could be to also a read a
<name-of-the-test-assembly>.specflow.json
file if it exists ?Steps to Reproduce
Create two test projects that need different specflow configurations and have the same output directory : only one of them can actually be properly configured.
Link to Repro Project
No response