chore: Fix tox not running locally

[wong-a]

Issue #, if available: N/A

Description of changes:

Fixes tox not running locally due to package version conflicts and updates CONTRIBUTING.md with steps to run integ tests.

tox was giving the following error before:

$ tox
GLOB sdist-make: /Users/wongada/workplace/aws-step-functions-data-science-sdk-python/setup.py
python3.6 create: /Users/wongada/workplace/aws-step-functions-data-science-sdk-python/.tox/python3.6
python3.6 inst: /Users/wongada/workplace/aws-step-functions-data-science-sdk-python/.tox/.tmp/package/1/stepfunctions-2.0.0.zip
ERROR: invocation failed (exit code 1), logfile: /Users/wongada/workplace/aws-step-functions-data-science-sdk-python/.tox/python3.6/log/python3.6-1.log
=================================================================== log start ===================================================================
Processing ./.tox/.tmp/package/1/stepfunctions-2.0.0.zip
Processing /Users/wongada/workplace/aws-step-functions-data-science-sdk-python
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
Collecting sagemaker>=2.1.0
  Using cached sagemaker-2.40.0-py2.py3-none-any.whl
Collecting boto3>=1.14.38
  Using cached boto3-1.17.73-py2.py3-none-any.whl
Collecting pyyaml
  Using cached PyYAML-5.4.1-cp38-cp38-macosx_10_9_x86_64.whl (253 kB)
INFO: pip is looking at multiple versions of stepfunctions to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install stepfunctions 2.0.0 (from /Users/wongada/workplace/aws-step-functions-data-science-sdk-python/.tox/.tmp/package/1/stepfunctions-2.0.0.zip) and stepfunctions[test]==2.0.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested stepfunctions 2.0.0 (from /Users/wongada/workplace/aws-step-functions-data-science-sdk-python/.tox/.tmp/package/1/stepfunctions-2.0.0.zip)
    stepfunctions[test] 2.0.0 depends on stepfunctions 2.0.0 (from /Users/wongada/workplace/aws-step-functions-data-science-sdk-python)

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

==================================================================== log end ====================================================================
____________________________________________________________________ summary ____________________________________________________________________
ERROR:   python3.6: InvocationError for command /Users/wongada/workplace/aws-step-functions-data-science-sdk-python/.tox/python3.6/bin/pip install --exists-action w .tox/.tmp/package/1/stepfunctions-2.0.0.zip '/Users/wongada/workplace/aws-step-functions-data-science-sdk-python[test]' (exited with code 1)

This seems to be triggered by the install_command line in tox.ini: https://github.com/aws/aws-step-functions-data-science-sdk-python/blob/fa979a4544e3701bf203c7dc2ee72934a20482d4/tox.ini#L12

I've modified it to use deps which seems like the normal way to specify dependencies for the virtual environment. https://tox.readthedocs.io/en/latest/config.html#conf-deps.

Usage in other packages:

• https://github.com/aws/sagemaker-python-sdk/blob/master/tox.ini#L72
• https://github.com/awslabs/aws-shell/blob/master/tox.ini#L6
• https://github.com/aws/aws-parallelcluster-cookbook/blob/develop/tox.ini#L10
• https://github.com/aws/aws-xray-sdk-python/blob/master/tox.ini#L17-L53

Across all public tox.ini files in GitHub, deps has 77,440 usages compared to 8,959 for install_command

The integ tests also require two IAM roles to give Step Functions and SageMaker permissions to execute actions on your behalf. I added instructions to create the roles in the CONTRIBUTING guide.

https://github.com/aws/aws-step-functions-data-science-sdk-python/blob/fa979a4544e3701bf203c7dc2ee72934a20482d4/tests/integ/conftest.py#L44-L50

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[ca-nguyen]

I'm still getting some permissions errors creating CloudWatch managed rules for some tests and have to sort that out.

Added instructions in the CONTRIBUTING guide to add missing CloudWatchEventsFullAccess policy to run tests successfully.

[wong-a]

I have one suggestion: can we link this contribution file to the Readme file? We can have link this doc in Readme's Table of contents so user can have a better entry point for local testing and contributing. It could be as easy as one line in the table and link to this doc.

Table of Contents should reflect the contents of the document - so don't just add something if it's not on the same page. Adding a section that links to the contribution guide would be good to have though. Some examples:

• https://github.com/aws/aws-cdk/blob/master/README.md#contributing
• https://github.com/aws/aws-sam-cli#contribute-to-sam

[StepFunctions-Bot]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildProject6AEA49D1-sEHrOdk7acJc
• Commit ID: bbdcab56b1ce76f21016d4c27750487b383368b8
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository