search

kondratyev-nv / vscode-python-test-adapter

Python Test Adapter for the VS Code Test Explorer
https://marketplace.visualstudio.com/items?itemName=LittleFoxTeam.vscode-python-test-adapter
MIT License
117 stars 27 forks source link

add support for behave BDD test framework #278

Open oberhofer opened 2 years ago

oberhofer commented 2 years ago

I want to add support for the behave BDD test framework. I like the clean structure of your project and I hope I got everything covered to support behave (https://github.com/behave/behave).

kondratyev-nv commented 2 years ago

Hi! Thanks for the PR! It seems some lint errors are still there. If you want to reproduce this locally, you can run npm run lint. I'll be reviewing your changes this week.

oberhofer commented 2 years ago

I will check the lint errors. In addition I'm not satisfied with my (non existent) behave test suite enumeration. Currently it only works for one test suite in the top level directory. Maybe you could tell me more about how this is supposed to work. Behave test suites have a dedicated "features" directory containing *.feature files. So in theory I have to scan a project for all directories, that contain a "features" directory and call behave there to get a json file with the test results for this suite. If you call behave in a directory without a "features" subdir you get a "ConfigError". Do I have to implement this scheme in load/run or is there another entity which is meant to scan directories ?

kondratyev-nv commented 2 years ago

I'm not familiar with behave, but I see 2 options there.

  1. You can try to replicate behave CLI behavior. As behave does not search for 'feature' files by itself, let the user specify what folders to include (with 'python.testing.behaveArgs' setting). This way it'd be straightforward on how to configure the extension if you previously called behave with some arguments like 'behave folder1 folder2', IMO it'd be rather obvious to set 'python.testing.behaveArgs' to [" folder1", "folder2"].
  2. Another option is to actually scan folders for 'feature' files. You can scan it and save it in the private property of the adapter code, see something similar here - https://github.com/kondratyev-nv/vscode-python-test-adapter/blob/master/src/pythonTestAdapter.ts#L76. Then pass this list to the testRunner somehow (idk, maybe another property in config object?) and then construct a list of behave arguments based on that.

IMO it's good to start with option 1 as it's simple. If it'll work and you'd feel like changing it afterward, you can upgrade the code following option 2. Let me know if this makes sense and what are your thoughts on this.

kondratyev-nv commented 2 years ago

btw, can you please merge the latest master brach to your PR, it should fix most of the CI and test errors.

oberhofer commented 2 years ago

IMO it's good to start with option 1 as it's simple. If it'll work and you'd feel like changing it afterward, you can upgrade the code following option 2. Let me know if this makes sense and what are your thoughts on this.

Sounds good. I will go with option 1 as behave supports positional arguments where you can specify either feature directories, single feature files or specific features with file:linenumber.

oberhofer commented 2 years ago

Ok, seems to work at least on my side, and I can specify multiple locations for my tests. Thanks for your support.