search

thevahidal / soul

🕉 A SQLite REST and realtime server
https://thevahidal.github.io/soul/
MIT License
1.45k stars 49 forks source link

Regression Testing of REST API #106

Open IanMayo opened 12 months ago

IanMayo commented 12 months ago

Is your feature request related to a problem? Please describe.

At this early point in its life Soul is going through lots of changes. It would be quite easy for the addition of a new feature, or implementation of a bug-fix to break some existing part of the API.

Soul developers could work with more confidence if there was an automated test-suite that verified the full API worked.

Describe the solution you'd like

Introduction of a GitHub action that runs a test-site across the full Soul REST API. This would run in each Pull Request, and give the maintainer(s) confidence that a new feature (or bug-fix) doesn't break any existing functionality.

Since Soul is a js project, a library such as Frisby may be an obvious choice.

I guess we'd build up a chinook.db instance that could be used for the unit testing. This would grow as required. So, when we were developing/reviewing the export views feature, we would have added a SQL VIEW to the database, to check it supported views.

Any other candidates for a library to run unit tests on the Soul REST API:?

Describe alternatives you've considered

Additional context Add any other context or screenshots about the feature request here.

IanMayo commented 12 months ago

Ok, I've found this existing test content: https://github.com/thevahidal/soul/blob/main/core/src/controllers/rows.test.js

This may have fallen behind the growth of new features, but we should be able to update/extend this test suite, and include it as a Pull Request action.

thevahidal commented 11 months ago

Yeah, you're completely right, Soul has tests right now but as you mentioned they might fall behind. But to run tests in the pipeline is a must-have and I'll try to do it asap.

thevahidal commented 11 months ago

We should have an strict acceptance criteria to enforce tests on newly added features, and also should write tests for the parts that lack any.

IanMayo commented 11 months ago

Yeah, you're completely right, Soul has tests right now but as you mentioned they might fall behind. But to run tests in the pipeline is a must-have and I'll try to do it asap.

Here is how to do it in a GitHub action: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs

You'd then go into repo settings / branches, create a new config for develop, and require that status checks are complete.

thevahidal commented 11 months ago

Thanks for the link, I'll make it a priority.