Extend and Align Tests of Operators

[JakobEliasWagner]

Description

Currently, unit and integration tests for operators are not aligned, leading to inconsistencies in testing methodologies and coverage. This issue proposes the establishment of a unified testing framework for all operators to ensure consistency, reliability, and comprehensive coverage of tests across our machine learning model operators.

Proposed Solution

Implement a standardized set of unit and integration tests that all operators must pass. This standardization includes:

• Common Unit Tests for Machine Learning Models: Define an extensive list of unit tests applicable to all machine learning model operators, focusing on functionality, performance, and edge cases.
• Single Integration Test: A single, integration test in the form of a convergence study on a specific dataset.

Expected Benefits

• Consistency: Ensuring all operators adhere to a unified testing standard will promote consistency across the board.
• Reliability: Comprehensive testing will significantly reduce the likelihood of bugs and errors, enhancing the overall reliability of the operators.
• Efficiency: Having a template and pytest fixtures in place to test these properties allows for easier, and faster implementation of tests for new operators.

Implementation Steps

1. Review Existing Tests.
2. Define Standardized Tests covering multiple different shapes and properties.
3. Implement Test Framework: Develop the testing framework according to the defined standards.
4. Documentation: Document the testing framework and guidelines for its use.
5. Refactor Existing Operators: Refactor current operators to comply with the new testing standards, ensuring all operators pass the defined tests.

Open Questions

• Selection of Dataset for Integration Test: What criteria should we use to select the dataset for the convergence study?