Supporting Scheduled Sagemaker Notebooks in VPC Mode

[TVAO]

The current cloud formation template does not support running Sagemaker Processing jobs within a VPC.

To solve this, the scheduling lambda function should be passed VpcConfig from the Sagemaker Studio domain if present.

Possibly, it should be passed to the InvokeNotebookLambda before being passed to the boto3 SDK call to Sagemaker create_processing_job, which ensures the Sagemaker processing instance has access to VPC resources.

Also, I believe the policy might need certain permissions to ec2 although the ones I have listed might be too many.

`Policies:

• PolicyName: root PolicyDocument: Version: 2012-10-17 Statement:
  • Effect: Allow Action:
    • sagemaker:CreateProcessingJob
    • iam:PassRole
    • ec2:DeleteNetworkInterface
    • ec2:CreateNetworkInterface
    • ec2:CreateNetworkInterfacePermission
    • ec2:DeleteNetworkInterface
    • ec2:DeleteNetworkInterfacePermission
    • ec2:DescribeNetworkInterfaces
    • ec2:DescribeVpcs
    • ec2:DescribeDhcpOptions
    • ec2:DescribeSubnets
    • ec2:DescribeSecurityGroups Resource: '*' InvokeNotebookLambda: Type: AWS::Lambda::Function Properties: FunctionName: RunNotebook Description: A function to run Jupyter notebooks using SageMaker processing jobs Handler: index.lambda_handler Runtime: python3.7 Role: {"Fn::GetAtt" : ["LambdaExecutionRole", "Arn"] } Timeout: 30 VpcConfig: SecurityGroupIds:
      • 'sg-' SubnetIds:
      • 'subnet-'
      • 'subnet-<xxxx'`

The VpcConfig should be passed into the boto3 create_processing_job request in the invoked lambda function:

"NetworkConfig": { "EnableInterContainerTrafficEncryption": boolean, "EnableNetworkIsolation": boolean, "VpcConfig": { "SecurityGroupIds": [ "string" ], "Subnets": [ "string" ] } }

In the code of the lambda template:

B['NetworkConfig'] = {'EnableInterContainerTrafficEncryption': False,'EnableNetworkIsolation': False,'VpcConfig': {'SecurityGroupIds': ['sg-<xxxx>'],'Subnets': ['subnet-<xxxx>','subnet-<xxxx>']}} V=boto3.client('sagemaker');H=V.create_processing_job(**B);W=H['ProcessingJobArn'];X=re.sub('^.*/','',W);return X

As of now, Scheduled Sagemaker Notebooks running Sagemaker Processing Jobs on Sagemaker Processing Instances do not have access to private resources within the VPC attached to Sagemaker Studio upon its domain creation.

[lasdem]

Thank you! I managed to get it working with running the job via command line by passing the Networking Config as --extra --extra '{ "NetworkConfig": { "EnableInterContainerTrafficEncryption": false, "EnableNetworkIsolation": false, "VpcConfig": { "SecurityGroupIds": [ "sg-xxx" ], "Subnets": [ "subnet-xxx" ] } } }'

I would really like if the UI would pass the vpc config from the sagemaker domain automatically.