The pull/75 change introduced AWS_REGION variables, but they are not reflected in unit test
% pipenv run python -m pytest -s -vv test/unit
...
E Left contains 2 more items:
E {'spark.executorEnv.AWS_REGION': 'None',
E 'spark.yarn.appMasterEnv.AWS_REGION': 'None'}
...
FAILED test/unit/test_bootstrapper.py::test_get_yarn_spark_resource_config - assert {'spark.default.parallelism': '8',\n 'spark.driver.defaultJavaOptions': "-XX:OnOutOfMemoryError='kill -9 %p' "\n ...
Description of changes:
I don't like just update assertion to have string "None", so I'm patching the os variable to have a reasonable value and then asserting that the value made it to the spark properties.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Issue #, if available:
The pull/75 change introduced
AWS_REGION
variables, but they are not reflected in unit testDescription of changes:
I don't like just update assertion to have string
"None"
, so I'm patching the os variable to have a reasonable value and then asserting that the value made it to the spark properties.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.