[Bug] "Cannot read property 'substring' of undefined" when launching workspace

awslabs / service-workbench-on-aws

A platform that provides researchers with one-click access to collaborative workspace environments operating across teams, universities, and datasets while enabling university IT stakeholders to manage, monitor, and control spending, apply security best practices, and comply with corporate governance.

Apache License 2.0

178 stars 119 forks source link

[Bug] "Cannot read property 'substring' of undefined" when launching workspace #961

Open tdmalone opened 2 years ago

tdmalone commented 2 years ago

Describe the bug When starting a workspace, the workspace fails to create with the error "Cannot read property 'substring' of undefined".

To Reproduce Steps to reproduce the behavior:

Go to Workspaces
Click on Create Research Workspace
Select a Compute option, eg. EC2 Linux-v1
Enter a name, choose a project and configuration, and enter a description
Click Create Research Workspace
Wait 2-3 minutes for the workspace to switch from Pending to Failed, with the above error

Expected behavior Expected the workspace to launch, or - for a more specific or detailed error to be returned.

Screenshots

Versions:

5.0.0

Additional context This is on a fresh install of Service Workbench 5.0.0, however I did add a new version (provisioning artifact) to the relevant Service Catalog product, with a modified volume size (as suggested at https://github.com/awslabs/service-workbench-on-aws/issues/866#issuecomment-1024572890). Launches are failing to work however with both the new and old template - they seem to not be making it to Service Catalog, or CloudFormation, at all.

tdmalone commented 2 years ago

Ah - I realised how I can track it down, at least. It appears to be coming from this line: https://github.com/awslabs/service-workbench-on-aws/blob/33b53da9a2793b5823bc979301468500d4f01289/addons/addon-environment-sc-api/packages/environment-sc-workflow-steps/lib/steps/check-launch-dependency/check-launch-dependency.js#L319

based on this fail state in the ...-workflow Step Function:

      "error": {
        "stpIndex": 4,
        "stpTmplId": "st-check-launch-dependency",
        "stpTmplVer": 1,
        "wfInstId": "7X8PNwK0PDuu2_1ZTe7JM",
        "wfId": "wf-provision-environment-sc",
        "wfVer": 1,
        "msg": "Cannot read property 'substring' of undefined",
        "stack": "TypeError: Cannot read property 'substring' of undefined\n    at CheckLaunchDependency.parseS3DetailsfromUrl (/var/task/src/lambdas/workflow-loop-runner/webpack:/home/ec2-user/service-workbench-v5.0.0/addons/addon-environment-sc-api/packages/environment-sc-workflow-steps/lib/steps/check-launch-depend"
      }

So it looks like this might be caused by some assumptions made in this regex: https://github.com/awslabs/service-workbench-on-aws/blob/33b53da9a2793b5823bc979301468500d4f01289/addons/addon-environment-sc-api/packages/environment-sc-workflow-steps/lib/steps/check-launch-dependency/check-launch-dependency.js#L310 in particular, that the S3 template URL is always going to follow that exact pattern. However, there are multiple patterns it could follow - at least https://s3.amazonaws.com/bucket-name/path/to/key.yaml and https://bucket-name.s3.amazonaws.com/path/to/key.yaml.

maghirardelli commented 2 years ago

Hi @tdmalone ! Thank you for reaching out and providing so many details about this bug. Would you be able to share the templateUrl that is being passed to this method that is causing it to fail? I have not been able to reproduce the error, but believe that the custom SC provisioning artifact is what is leading to this error. The templateUrl is being queried from the SC Client in https://github.com/awslabs/service-workbench-on-aws/blob/33b53da9a2793b5823bc979301468500d4f01289/addons/addon-environment-sc-api/packages/environment-sc-workflow-steps/lib/steps/check-launch-dependency/check-launch-dependency.js#L265-L276

You can check what the method above is returning by using the aws cli and running this command after configuring your .aws/config and .aws/credentials files for a certain profile (or default if you only have one profile in each file). Replace the elements in <> with your values. If you are using your default profile, the --profile param is not required:

aws servicecatalog describe-provisioning-artifact --provisioning-artifact-id pa-<id> --product-id prod-<id> --profile <profilename>

My response looks like this:

{
    "ProvisioningArtifactDetail": {
        "Id": "pa-<id>",
        "Name": "v3",
        "Description": "* An EC2 Linux instance with SSH access \n* Secure compute in the cloud",
        "Type": "CLOUD_FORMATION_TEMPLATE",
        "CreatedTime": 1642531767.0,
        "Active": true,
        "Guidance": "DEFAULT"
    },
    "Info": {
        "TemplateUrl": "https://<account-id>-<stage>-<region-short-name>-<solution-namespace>-artifacts.s3.amazonaws.com/service-catalog-products/ec2-linux-instance.cfn.yml"
    },
    "Status": "AVAILABLE"
}

Your input will help me debug if our check in the method you linked above is correct or not. Thank you!

tdmalone commented 2 years ago

Hi @maghirardelli, sorry for the delay in getting back to you on this!

The template URL I had looked like:

https://s3.amazonaws.com/<account-id>-<stage>-<region-short-name>-<solution-namespace>-artifacts/service-catalog-products/ec2-linux-instance.cfn.v2.yml

nguyen102 commented 2 years ago

@tdmalone I think you're right, with your root cause. I think the issue is the expected S3 URL format. I tested the following code and was able to reproduce the error.

const url = require('url');
// const stuff = ['a', 'b', 'c'];
//
// for (letter of stuff) {
//     if (letter === 'b') {
//         continue;
//     }
//     console.log(letter);
// }

function parseS3DetailsfromUrl(templateUrl) {
    const ENDPOINT_PATTERN = /^(.+\.)?s3[.-]([a-z0-9-]+)\./;
    const uri = url.parse(templateUrl);
    const matches = uri.host.match(ENDPOINT_PATTERN);
    if (!matches) {
        throw new Error(`Invalid S3 URL: ${templateUrl}`);
    }
    const prefix = matches[1];
    // remove last . from prefix
    const bucketName = prefix.substring(0, prefix.length - 1);
    const key = uri.pathname.substring(1);
    return { bucketName, key };
}
const workingS3Url = "https://0123456789012-dev-va-swb-artifacts.s3.amazonaws.com/service-catalog-products/ec2-linux-instance.cfn.yml"
// const s3UrlThatResultInError = "https://s3.amazonaws.com/0123456789012-dev-va-swb-artifacts/service-catalog-products/ec2-linux-instance.cfn.v2.yml"

console.log(parseS3DetailsfromUrl(workingS3Url));

I'm curious as to why your S3 url is different from the expected format. When I tested my deployment, my url was also in the expected format "https://<account-id>-<stage>-<region-short-name>-<solution-namespace>-artifacts.s3.amazonaws.com/service-catalog-products/ec2-linux-instance.cfn.yml".

Did you add any custom setup to your SWB deployment?

tdmalone commented 2 years ago

Did you add any custom setup to your SWB deployment?

I did, yes - see the 'additional context' section in my original report above.