malexer / pytest-spark

pytest plugin to run the tests with support of pyspark
MIT License
85 stars 30 forks source link

Unable to use it in a unittest.TestCase #6

Closed AbdealiLoKo closed 6 years ago

AbdealiLoKo commented 6 years ago

Was trying to use this in my test cases. I normally use the Class based approach with unittest.TestCase and was not able to get it to work. Example:

class ExampleTest(unittest.TestCase):

    @pytest.mark.usefixtures("spark_session")
    def test_create_df(self, spark_session):
        df = spark_session.createDataFrame([
            [1, 1, 1],
            [1, 2, 1],
            [1, 2, 4]
        ], columns=['x', 'y', 'z'])

I can see that pyspark recognized the plugin, the spark session has been created but it fails with: TypeError: test_create_df() missing 1 required positional argument: 'spark_session'

malexer commented 6 years ago

This plugin (pytest-spark) is specifically for usage with pytest framework and is dependent on its features.

Please, check the note section in the end of this page:

While pytest supports receiving fixtures via test function arguments for non-unittest test methods, unittest.TestCase methods cannot directly receive fixture function arguments as implementing that is likely to inflict on the ability to run general unittest.TestCase test suites. ...

For your case, you can consider using my other plugin for the nose2: nose2-spark