Spark and python's evaluation metrics should produce the same result.
For that, we should refactor pytest to check if they produce the same values for the same test.
This issue is to track the pytest refactoring to implement that.
Expected behavior with the suggested feature
Define test parameters and expected-outputs by using a fixture. Then use the same fixture for both python and pyspark evaluation metrics.
Description
Spark and python's evaluation metrics should produce the same result. For that, we should refactor pytest to check if they produce the same values for the same test. This issue is to track the pytest refactoring to implement that.
Expected behavior with the suggested feature
Define test parameters and expected-outputs by using a fixture. Then use the same fixture for both python and pyspark evaluation metrics.