The current test coverage needs to be improved because currently the integration tests only covers a specific spark version (3.1). Insufficient coverage to other supported Spark versions could cause the problem like incompatible Spark versions however these issues should be captured before release.
Steps to Reproduce
None
Expected Behavior
The test workflow should run against different versions of Spark to ensure that new changes do not introduce version incompatible changes.
Actual Behavior
For now the integration tests only run against 1 Spark version.
Screenshots
None
Environment
Operating System:
Browser:
Version of the software:
Additional Information
Any other information that may be helpful in debugging the issue.
Summary
The current test coverage needs to be improved because currently the integration tests only covers a specific spark version (3.1). Insufficient coverage to other supported Spark versions could cause the problem like incompatible Spark versions however these issues should be captured before release.
Steps to Reproduce
None
Expected Behavior
The test workflow should run against different versions of Spark to ensure that new changes do not introduce version incompatible changes.
Actual Behavior
For now the integration tests only run against 1 Spark version.
Screenshots
None
Environment
Additional Information
Any other information that may be helpful in debugging the issue.