Failing unit tests (setup steps required?)

PopMedNet-Team / popmednet

This is the repository for the open source code of the distributed data network platform PopMedNet

https://www.popmednet.org/

Other

11 stars 9 forks source link

Failing unit tests (setup steps required?) #38

Closed lrasmus closed 2 years ago

lrasmus commented 2 years ago

As part of https://github.com/openjournals/joss-reviews/issues/4062, I am running through unit tests and had multiple failures. The error is fairly consistent and descriptive:

    Unable to create instance of class Lpp.Dns.DataMart.Model.QueryComposer.Tests.SummaryQueryTests. Error: System.Configuration.ConfigurationErrorsException: Unable to open configSource file 'ConnectionStrings.config'. (C:\Development\popmednet\Lpp.Adapters\Lpp.Dns.DataMart.Model.QueryComposer.Tests\bin\Lpp.Dns.DataMart.Model.QueryComposer.Tests.dll.config line 97).

I wanted to confirm (I did not see documentation on the README page) what setup steps are needed to run the unit tests. Are these more integration tests - requiring a full database setup, and setup of a configuration file?

ddeehere commented 2 years ago

They do need a database setup with sample CDM (common data model) data. The unit tests do not make any modification to the database. You will need to be provided with this test data. We are planning to submit that to IEEE Data Port. In the meantime, I will figure out how to provide it to you.

The ConnectionString.config will have to be copied from the template and modified accordingly.

lrasmus commented 2 years ago

Thank you @ddeehere! Do the tests expect your specific dataset, or is it possible to incorporate - for example - an OMOP CDM populated with SynPuf data? If there are general instructions (apologies if I missed these) on how to configure and run the tests, I'm happy to keep proceeding.

ddeehere commented 2 years ago

@lrasmus I added a section on Regression Tests in the README.md, which links to the Confluence page with an explanation of the framework and the steps to run it. The way the tests is collected is this:

New features are manually tested.
Once verified, the query and the results (request and response) are collected as JSON files and added to a folder called Resources, and a new test method is added as well. Normally, a test method simply calls a common method in the superclass.
When run, the request file is loaded and run through the engine. The outcome is then compared against the response file.
If there is a discrepancy, then we know that either the data has changed or a bug has been introduced.

Currently, we do not support OMOP.

NOTE: Still working on providing the CDM.

lrasmus commented 2 years ago

@ddeehere - thank you. Could you please contact me at luke.rasmussen[at]gmail.com about this latest commit?