Closed varun196 closed 3 years ago
Description
EMR connection will fail if master node is not retrieved in the non-paginated list_instances call. This commits paginates the call, reformats the file using black and bounces PyPI version.
black
resolves #2
Testing Done
Added unit test.
❯ python3 -m black . && python3 -m pytest -vrfpP . ─╯ All done! ✨ 🍰 ✨ 10 files left unchanged. ===================================================================================== test session starts ===================================================================================== platform linux -- Python 3.7.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /workplace/shahvar/Astra/src/LooseLeafSparkMagicImage/SMStudioSparkMagicLib plugins: anyio-3.3.0 collected 12 items sagemaker_studio_sparkmagic_lib/tests/emr_test.py::test_get_cluster_happy_case_non_kerberos PASSED [ 8%] sagemaker_studio_sparkmagic_lib/tests/emr_test.py::test_get_cluster_happy_case_non_kerberos_with_pagination PASSED [ 16%] sagemaker_studio_sparkmagic_lib/tests/emr_test.py::test_get_cluster_happy_case_kerberos PASSED [ 25%] sagemaker_studio_sparkmagic_lib/tests/emr_test.py::test_get_bad_cluster PASSED [ 33%] sagemaker_studio_sparkmagic_lib/tests/emr_test.py::test_cluster_dedicated_krb_cluster PASSED [ 41%] sagemaker_studio_sparkmagic_lib/tests/emr_test.py::test_cross_realm_krb_cluster PASSED [ 50%] sagemaker_studio_sparkmagic_lib/tests/emr_test.py::test_external_kdc_cluster PASSED [ 58%] sagemaker_studio_sparkmagic_lib/tests/kerberos_test.py::test_generated_krb_conf PASSED [ 66%] sagemaker_studio_sparkmagic_lib/tests/utils_test.py::test_get_domain_search_without_search_line PASSED [ 75%] sagemaker_studio_sparkmagic_lib/tests/utils_test.py::test_get_domain_search_happy_case PASSED [ 83%] sagemaker_studio_sparkmagic_lib/tests/utils_test.py::test_get_domain_search_resolv_does_not_exist PASSED [ 91%] sagemaker_studio_sparkmagic_lib/tests/utils_test.py::test_get_emr_endpoint_url PASSED [100%]
Also verified by installing on Studio and trying to connect to a cluster with 100 instances.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Can you bounce the version and also add description in the changelog?
Description
EMR connection will fail if master node is not retrieved in the non-paginated list_instances call. This commits paginates the call, reformats the file using
black
and bounces PyPI version.resolves #2
Testing Done
Added unit test.
Also verified by installing on Studio and trying to connect to a cluster with 100 instances.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.