Error while running ./run-all-tests.sh

[progerjkd]

Describe the bug I cloned the aws-mlops-framework repository into my SageMake Studio, then I ran ./run-all-tests.sh which failed when testing test_lambda_function.py.

To Reproduce git clone https://github.com/awslabs/aws-mlops-framework.git cd aws-mlops-framework/source ./run-all-tests.sh

Expected behavior Script run-all-tests.sh to finish successfully

Please complete the following information about the solution:

• [ ] Version: v1.3.0

To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "(SO0136) - AWS MLOps Framework. Version v1.0.0".

• [ ] Region: [ca-central-1]
• [ ] Was the solution modified from the version published on this repository? No
• [ ] If the answer to the previous question was yes, are the changes available on GitHub?
• [ ] Have you checked your service quotas for the sevices this solution uses?
• [ ] Were there any errors in the CloudWatch Logs? No

Screenshots

ERROR: jinja2 3.0.1 has requirement MarkupSafe>=2.0, but you'll have markupsafe 1.1.1 which is incompatible.
test_lambda_function.py .....F                                                                                                                                         [100%]

================================================================================== FAILURES ==================================================================================
__________________________________________________________________ LambdaTest.test_send_metrics_successful ___________________________________________________________________

self = <test_lambda_function.LambdaTest testMethod=test_send_metrics_successful>, mock_post = <MagicMock name='post' id='140431262800912'>

    @mock.patch("requests.post", side_effect=mocked_requests_post)
    def test_send_metrics_successful(self, mock_post):
        event = {
            "RequestType": "Create",
            "ResourceProperties": {
                "Resource": "AnonymousMetric",
                "SolutionId": "SO1234",
                "gitSelected": "True",
                "bucketSelected": "False",
                "UUID": "some-uuid",
                "Foo": "Bar",
            },
        }

        from lambda_function import custom_resource

        custom_resource(event, None)

        expected_metrics_endpoint = "https://metrics.awssolutionsbuilder.com/generic"
        actual_metrics_endpoint = mock_post.call_args.args[0]
>       self.assertEqual(expected_metrics_endpoint, actual_metrics_endpoint)
E       AssertionError: 'https://metrics.awssolutionsbuilder.com/generic' != 'args'
E       - https://metrics.awssolutionsbuilder.com/generic
E       + args

test_lambda_function.py:58: AssertionError
----------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------
INFO     lambda_function:lambda_function.py:61 Sending payload: {'Solution': 'SO1234', 'UUID': 'some-uuid', 'TimeStamp': '2021-07-02T16:40:21.960579', 'Data': {'gitSelected': 'True', 'bucketSelected': 'False', 'Foo': 'Bar', 'RequestType': 'Create'}}
INFO     lambda_function:lambda_function.py:63 Response from metrics endpoint: 200 OK

---------- coverage: platform linux, python 3.7.10-final-0 -----------
Name                 Stmts   Miss  Cover   Missing
--------------------------------------------------
lambda_function.py      34      0   100%
--------------------------------------------------
TOTAL                   34      0   100%
Coverage XML written to file /home/sagemaker-user/aws-mlops-framework/source/test/coverage-reports/solution_helper.coverage.xml

Required test coverage of 80% reached. Total coverage: 100.00%
========================================================================== short test summary info ===========================================================================
FAILED test_lambda_function.py::LambdaTest::test_send_metrics_successful - AssertionError: 'https://metrics.awssolutionsbuilder.com/generic' != 'args'
======================================================================== 1 failed, 5 passed in 2.85s =========================================================================

Additional context Add any other context about the problem here.

[tabdunabi]

Hi @progerjkd, thank you for reaching out and opening this ticket.

The Prerequisite section outlines what is needed to successfully build the solution. Specifically, Python 3.8 and Docker. We have tested building the solution using this setup.

SageMaker Studio has Python 3.7.10 (you can open a terminal and run python --version to verify the version) and does not have Docker installed by default.

You do not need SageMaker studio to build the solution. So, you can setup the required environment on a local machine or EC2 instance and build the solution.

If you are not making any custom code changes, you do not have to build the solution by yourself to try the solution. You can access pre-built assets from the landing page of the AWS MLOps Framework solution by deploying the AWS CloudFormation template using Launch in the AWS Console using Option 1 (or 2).

[progerjkd]

Thanks for the explanation.