Add Spark EMR periodic tests

[hsubbaraj-spiral]

Describe your changes and why you are making these changes

Add Spark EMR periodic tests

Related issue number (if any)

Eng-2882 and Eng-2883

Loom demo (if any)

Checklist before requesting a review

• [x] I have created a descriptive PR title. The PR title should complete the sentence "This PR...".
• [x] I have performed a self-review of my code.
• [ ] I have included a small demo of the changes. For the UI, this would be a screenshot or a Loom video.
• [ ] If this is a new feature, I have added unit tests and integration tests.
• [ ] I have run the integration tests locally and they are passing.
• [ ] I have run the linter script locally (See python3 scripts/run_linters.py -h for usage).
• [ ] All features on the UI continue to work correctly.
• [x] Added one of the following CI labels:
  • run_integration_test: Runs integration tests
  • skip_integration_test: Skips integration tests (Should be used when changes are ONLY documentation/UI)