search

malexer / pytest-spark

pytest plugin to run the tests with support of pyspark
MIT License
84 stars 30 forks source link

Bugfix #10 - Support missing spark_options from pytest.ini #12

Closed skyegecko closed 5 years ago

skyegecko commented 5 years ago

Hi,

This PR includes a test-case and fix for #10. It depends on #11.

malexer commented 5 years ago

@dutchgecko have introduced some changes, please review

malexer commented 5 years ago

still need to deal with Travis config, will do it later

skyegecko commented 5 years ago

The tox config is obviously better, so that's fine.

Your version in 2ef93f4 gives a smaller diff (no changes to pytest_spark/__init__.py) and doesn't need to defensively raise like I added, but will mean that initialize() can be called from different locations. I've got no problems with this, and it's your project so you can arrange the code as you like, just pointing out the tradeoff.

malexer commented 5 years ago

will mean that initialize() can be called from different locations

yep, idea was to have SparkConfigBuilder as a singleton

malexer commented 5 years ago

Released in 0.5.2. Please find it on pypi