mathworks / matlab-azure-devops-extension

Continuous Integration with MATLAB on Azure DevOps
https://marketplace.visualstudio.com/items?itemName=MathWorks.matlab-azure-devops-extension
Other
14 stars 5 forks source link

Tests filtered out because of missing classes at init time #49

Open pgeschwill opened 3 years ago

pgeschwill commented 3 years ago

Hi,

thanks for providing this plugin. It is a great improvement over having to write your own powershell code for running MATLAB on DevOps agents.

I have encountered a couple of issues which might be a basis for an enhancement of a future release.

1.) By default, when running MATLAB tests with the plugin, it first scans the root folder for test files. In my case, there are a bunch of test helper classes which are used during test setup and execution. Some are simple enums, some are more complex classes which handle setting up a SQL database before a test. The tests using these classes are automatically filtered out when executing the tests because the MATLAB session does not know the classes when setting up the suite.

First of all, is there currently a more elegant way to work around this than just using a RunMATLABCommand@0 which runs a script that adds the classes to the search path?

It would be nice to have an additional setting for RunMATLABTests@0 where you can specify code that should be executed before setting up the suite such that you can add helper classes to the path.

2.) When specifying a selectByFolder option in RunMATLABTests@0 and passing in an absolute path like "C:\Users\Public" on a Windows agent, this leads to two paths being added to the search path: "C" and "Users\Public" due to the colon separation feature. I have not found a way to work around this on Windows machines. Would it be an option to remove that feature and instead only allowing semicolons as separators?

I'd be happy to discuss these points with you.

Best Pascal

acampbel commented 3 years ago

Thank you Pascal for reporting this issue.

For your first question, have you considered using a MATLAB project? This is a way that you can more explicitly setup your project development environment. Can you check this out and confirm whether this will help you in your use case?

https://blogs.mathworks.com/developer/2020/06/11/project-yourself/

https://blogs.mathworks.com/developer/2020/08/19/tests-in-projects/

For your second question I agree that is not ideal. Let me discuss this with the development team so we can explore how we can address the problem.

Thanks! Andy

mcafaro commented 3 years ago

selectByFolder was originally intended for relative paths so you can filter down the tests in your repository. Can you let us know how you'd like to use it with an absolute path? Is your intent to add some additional tests outside of your repository?

acampbel commented 2 years ago

Hi @pgeschwill,

Is this still an issue for you? Did using projects work for your use case? If not you can also leverage a startup.m file in the root of your repo where you launch matlab from which can add the necessary paths. Projects aim to be a solution that would work for your development environment as well.

Another workaround for the issue with the separators issue is to use UNC paths instead of mapped drive letters if that is possible. Can you lend any insight into why you are using absolute as opposed to relative paths?

Thanks, Andy