a-jaxell commented 1 day ago

Integration testing

To make sure functionality works with multiple cases an integration test for each case would be a good way to prevent new features from breaking current functionality.

Definition

Multifile repository - A repository with multiple .yml files containing translations. Singular file repository - A repository with a single .yml file containing translations.

Use cases

As a user I want to have translations across multiple files in my repo
As a user I want to have translations in a singular file in my repo

Tasks

[ ] Integration test for a singular file structure
[ ] Integration test for a multifile repository

a-jaxell commented 1 day ago

I would love to get feedsback on how to tackle this issue @WULCAN @amerharb

Want to dive into this and make tests for both cases. Is there something I have missed or forgot to think about?

WULCAN commented 14 hours ago

We corresponded about this over Slack yesterday and I'll try to summarize here.

I don't think you missed anything with this issue. I imagine there will be a lot of different scenarios we will want to cover with integration tests but even just one or a couple of scenarios to begin with would be a great improvement.

We currently have Jest and while it seems more geared towards unit tests, I think it could possibly work well for larger integrationstests too. Different persons have different opinions on what an integration test is; I see it like a unit test with a much larger unit under test.

In app.zetkin.org, we also use Playwright, which is a very good framework for testing UI. If we want to include UI in the integration tests, I suggest we try Playwright instead of Jest.

If we don't want to include the UI, I think we will want to skip testing the HTTP layer as well, and instead exercise the data access layer and the server actions directly. We're not too worried about incorrect integration with Next right now.

I think we will also have to mock some dependencies of Lyra that I expect will be problematic to include in the unit under test:

file system: used to store and update local clones of the translated projects' repositories
git: used to clone, update, pull, commit and push to GitHub
GitHub: used to create pull requests

Even if we do mock them, it could get very verbose to mock these, as the interfaces are huge and it can be a little hard to know exactly what parts of the interfaces Lyra uses.

Note that we already have multiple tests using a kind of file system mock. See TsMessageAdapter and YamlMessageAdapter. Have a look at that system before introducing a new one, it might be sufficient for our needs. There's also some kind of virtual file system available with one of our dependencies, but I don't remember if its the git library or the typescript library. It might be worthwhile to have a look at that.

zetkin / lyra