pontushenriksson
commented
3 months ago This is now merged with Issue #20. We will create a script that updates the contents of the CONTRIBUTORS.md file together with the data from GitHub and its workflows. This will not affect the performance of the compiler but rather make new contributors get acknowledged quicker which they deserve. More information about the stage we are at and our future implementation of this will come shortly in a follow-up comment here.
@lucasnorman07 have implemented a few tests inside the compiler so now this just needs to be connected to a new workflow.
Is your feature request related to a problem? Please describe.
Our current manual testing approach is prone to errors, and the lack of automated tests makes it difficult to ensure the stability and reliability of Lotus with each release. The ./tests directory setup remains unutilized, leaving us vulnerable to regressions that could disrupt our development cycle. The frustration of encountering preventable bugs is compounded by the absence of a systematic testing protocol.
Describe the solution you'd like
I propose the creation of a robust testing framework within the existing ./tests directory, comprising both unit and end-to-end tests. Unit tests will scrutinize the functionality of individual components, while end-to-end tests will assess the cohesive operation of the entire language in user-emulated scenarios. Crucially, we need to introduce a new GitHub workflow that automatically executes these tests against every commit, complementing our existing build and lint workflows. This will ensure that tests are an integral part of our CI/CD process, maintaining code integrity and quality.
Describe alternatives you've considered
Persisting with manual testing or partial automation (only unit tests) has been considered, but neither scales effectively nor offers comprehensive coverage. While test-driven development (TDD) is an excellent ongoing strategy, it does not address our immediate need for a full testing suite for existing code. Moreover, without a dedicated workflow to run these tests automatically, we cannot fully leverage the benefits of automated testing.
Additional context
A systematic testing suite is vital for advancing the Lotus language with confidence. It enables us to maintain high standards and gives contributors precise guidelines for verifying code quality. To streamline this process, we must establish a new GitHub workflow specifically for running tests housed in the ./tests directory. This addition will close the gap in our current CI/CD practices, providing a much-needed safety net for our codebase.