Add Error Handling for No Screenshots Generated in Visual Tests

Increasing access

In the visual testing framework, there is a scenario where visual tests can pass despite not generating any screenshots. This situation typically arises in tests that are designed to capture and compare screenshots. However, due to various factors, such as asynchronous operations not completing, these tests may end up generating no screenshots at all.

The core issue here is that the absence of generated screenshots leads to false positives in test results. Since no visual comparison is actually performed (due to the lack of screenshots), the test may incorrectly pass, giving the impression that the tested visuals meet the expected criteria when, in fact, this has not been verified. This can mask potential issues in the visuals that the test is supposed to validate.

The objective is to ensure that every visual test that is intended to produce screenshots for comparison actually does so. If a test fails to generate screenshots, it should not pass silently. Instead, it should alert the developer to the lack of visual output, indicating a potential problem in the test setup or execution.

Most appropriate sub-area of p5.js?

[ ] Accessibility
[ ] Color
[ ] Core/Environment/Rendering
[ ] Data
[ ] DOM
[ ] Events
[ ] Image
[ ] IO
[ ] Math
[ ] Typography
[ ] Utilities
[ ] WebGL
[ ] Build process
[X] Unit testing
[ ] Internationalization
[ ] Friendly errors
[ ] Other (specify if possible)

Feature enhancement details

In test/unit/visual/visualTest.js, adding error handling to address cases where actual.length === 0, indicating that no screenshots were generated during a visual test. This enhancement will throw an error under such circumstances, alerting the developer the absence of screenshots and that there might be a need to wait for a promise to ensure if the test had asynchronous operations, they are completed before the test ends.

processing / p5.js