search

GSA / 889-tool

Web service for determining 889 compliance of vendors
1 stars 0 forks source link

Failing tests in test suite #7

Closed mark-meyer closed 10 months ago

mark-meyer commented 1 year ago

The tests in /test/test_samtools.py hit the live SAM API and make assertions based on the returned results. Because the data in the SAM API is not constant, these tests can be fragile. A couple are now failing because the entities have changed.

The tests seem to be testing that the logic of the queries work against the SAM API, which is valuable, but it's not clear that tests hitting and live API belong in an automated test suite.

mark-meyer commented 1 year ago

In building the search parameters in /samtools/sam_api/search_preprocessor.py the code parenthesizes the entire search string. This produces unusual results and may be responsible for on for the failing tests.

Experimenting with the SAM api, it seems possible to fix one test without breaking others by changing the parenthesis to be: return {'q': f'(legalBusinessName:{business_name}) OR (dbaName:{business_name})'}

This however breaks a bunch of tests that make assertions based on the exact string in the query. It will take some more experimentation and edits of tests to change this.

mark-meyer commented 10 months ago

Addressed by removing flakey tests that depend on SAM api.