aws-samples / service-catalog-engine-for-terraform-os

Apache License 2.0
128 stars 40 forks source link

deploy-tre script fails with error "Error: GoModulesBuilder:Build - Builder Failed: stat /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/terraform_open_source_parameter_parser/terraform_open_source_parameter_parser: directory not found" #68

Open kaul-abhi opened 8 months ago

kaul-abhi commented 8 months ago

Hello Team, When i am running script for deployment and when command "sam build" runs it fails with error "Error: GoModulesBuilder:Build - Builder Failed: stat /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/terraform_open_source_parameter_parser/terraform_open_source_parameter_parser: directory not found".

Go version : go version go1.20.7 No changes is made in template.yaml for CodeUri section but still it fails. PFB the error logs:

$ sam --version SAM CLI, version 1.100.0 $ aws --version aws-cli/2.13.20 Python/3.11.5 Linux/6.0.7-301.fc37.x86_64 exe/x86_64.alpine.3 prompt/off $ python3.9 -V Python 3.9.13 $ pip3 --version pip 23.3.1 from /usr/lib/python3.11/site-packages/pip (python 3.11) $ export PATH=$PATH:/usr/local/go/bin $ which go /usr/local/go/bin/go $ go version go version go1.20.7 linux/amd64 $ /pipeline/treinstall $TRE_REPO_LOCATION = 'https://github.com/aws-samples/service-catalog-engine-for-terraform-os.git' AWS_REGION : eu-west-1 $AWS_REGION = eu-west-1 Clone the Terraform Reference Engine code from github. Cloning into 'service-catalog-engine-for-terraform-os'... 10:52:19 [INFO] [treinstall] cleaning template.yml file so to copy customized template.yaml copying customized template.yml file into TRE directory 10:52:19 [INFO] [treinstall] current profile information for troubleshooting in case of publish errors { "UserId": "AROAVRDUU2NP36G2CFAUV:GitLabRunner-327317-145418590", "Account": "380349240159", "Arn": "arn:aws:sts::380349240159:assumed-role/csc-runner-default/GitLabRunner-327317-145418590" } running the deploy-tre script AWS account: 380349240159 AWS region: eu-west-1 Creating a Python virtual environment Building the ServiceCatalogTerraformOSParameterParser function go: creating new go.mod: module terraform_open_source_parameter_parser go: to add module requirements and sums: go mod tidy go: finding module for package github.com/aws/aws-sdk-go/aws go: finding module for package github.com/aws/aws-sdk-go/aws/arn go: finding module for package github.com/stretchr/testify/mock go: finding module for package github.com/aws/aws-lambda-go/lambda go: finding module for package github.com/aws/aws-sdk-go/aws/credentials go: finding module for package github.com/aws/aws-sdk-go/aws/credentials/stscreds go: finding module for package github.com/aws/aws-sdk-go/aws/session go: finding module for package github.com/aws/aws-sdk-go/service/s3 go: finding module for package github.com/aws/aws-sdk-go/service/s3/s3manager go: finding module for package github.com/hashicorp/hcl/v2/hclparse go: finding module for package github.com/hashicorp/terraform-config-inspect/tfconfig go: downloading github.com/stretchr/testify v1.8.4 go: downloading github.com/hashicorp/terraform-config-inspect v0.0.0-20230925220900-5a6f8d18746d go: downloading github.com/aws/aws-lambda-go v1.41.0 go: downloading github.com/hashicorp/hcl/v2 v2.19.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/aws/aws-sdk-go v1.47.2 go: found github.com/aws/aws-lambda-go/lambda in github.com/aws/aws-lambda-go v1.41.0 go: found github.com/aws/aws-sdk-go/aws in github.com/aws/aws-sdk-go v1.47.2 go: found github.com/aws/aws-sdk-go/aws/arn in github.com/aws/aws-sdk-go v1.47.2 go: found github.com/aws/aws-sdk-go/aws/credentials in github.com/aws/aws-sdk-go v1.47.2 go: found github.com/aws/aws-sdk-go/aws/credentials/stscreds in github.com/aws/aws-sdk-go v1.47.2 go: found github.com/aws/aws-sdk-go/aws/session in github.com/aws/aws-sdk-go v1.47.2 go: found github.com/aws/aws-sdk-go/service/s3 in github.com/aws/aws-sdk-go v1.47.2 go: found github.com/aws/aws-sdk-go/service/s3/s3manager in github.com/aws/aws-sdk-go v1.47.2 go: found github.com/hashicorp/hcl/v2/hclparse in github.com/hashicorp/hcl/v2 v2.19.1 go: found github.com/hashicorp/terraform-config-inspect/tfconfig in github.com/hashicorp/terraform-config-inspect v0.0.0-20230925220900-5a6f8d18746d go: found github.com/stretchr/testify/mock in github.com/stretchr/testify v1.8.4 go: downloading github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f go: downloading github.com/zclconf/go-cty v1.13.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/google/go-cmp v0.3.1 go: downloading github.com/stretchr/objx v0.5.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/agext/levenshtein v1.2.2 go: downloading github.com/apparentlymart/go-textseg/v15 v15.0.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/kr/pretty v0.1.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/apparentlymart/go-textseg v1.0.0 go: downloading golang.org/x/net v0.17.0 go: downloading github.com/sergi/go-diff v1.0.0 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading golang.org/x/text v0.13.0 go: downloading github.com/kr/text v0.1.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 go: downloading gopkg.in/yaml.v2 v2.2.8 Building the Lambda code Collecting wheel Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/fa/7f/4c07234086edbce4a0a446209dc0cb08a19bb206a3ea53b2f56a403f983b/wheel-0.41.3-py3-none-any.whl.metadata Downloading wheel-0.41.3-py3-none-any.whl.metadata (2.2 kB) Downloading wheel-0.41.3-py3-none-any.whl (65 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.8/65.8 kB 674.9 kB/s eta 0:00:00 Installing collected packages: wheel Successfully installed wheel-0.41.3 [notice] A new release of pip is available: 23.2.1 -> 23.3.1 [notice] To update, run: pip install --upgrade pip Collecting boto3>=1.26.105 (from -r lambda-functions/state_machine_lambdas/requirements.txt (line 1)) Obtaining dependency information for boto3>=1.26.105 from https://files.pythonhosted.org/packages/72/01/eded6bd8244670f1609743a0e63e097d86d8f79801d323fc18b00267cf20/boto3-1.28.77-py3-none-any.whl.metadata Downloading boto3-1.28.77-py3-none-any.whl.metadata (6.7 kB) Collecting botocore>=1.29.105 (from -r lambda-functions/state_machine_lambdas/requirements.txt (line 2)) Obtaining dependency information for botocore>=1.29.105 from https://files.pythonhosted.org/packages/56/05/f8d8d52d15e713909facb6a13276822fea04da16e3d4de08c8860c9884c7/botocore-1.31.77-py3-none-any.whl.metadata Downloading botocore-1.31.77-py3-none-any.whl.metadata (6.1 kB) Collecting jmespath<2.0.0,>=0.7.1 (from boto3>=1.26.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 1)) Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) Collecting s3transfer<0.8.0,>=0.7.0 (from boto3>=1.26.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 1)) Obtaining dependency information for s3transfer<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/5a/4b/fec9ce18f8874a96c5061[42](https://code.siemens.com/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/-/jobs/145418590#L42)2625ba86c3ee1e6587ccd92ff9f5bf7bd91b2/s3transfer-0.7.0-py3-none-any.whl.metadata Downloading s3transfer-0.7.0-py3-none-any.whl.metadata (1.8 kB) Collecting python-dateutil<3.0.0,>=2.1 (from botocore>=1.29.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 2)) Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 4.9 MB/s eta 0:00:00 Collecting urllib3<2.1,>=1.25.4 (from botocore>=1.29.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 2)) Obtaining dependency information for urllib3<2.1,>=1.25.4 from https://files.pythonhosted.org/packages/d2/b2/b15785[51](https://code.siemens.com/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/-/jobs/145418590#L51)92a68541a91ba7b2bbcb91f1b4faa51f8bae38d8005c034be[52](https://code.siemens.com/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/-/jobs/145418590#L52)4/urllib3-2.0.7-py3-none-any.whl.metadata Downloading urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB) Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore>=1.29.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 2)) Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) Downloading boto3-1.28.77-py3-none-any.whl (135 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 135.8/135.8 kB 9.1 MB/s eta 0:00:00 Downloading botocore-1.31.77-py3-none-any.whl (11.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.3/11.3 MB 36.7 MB/s eta 0:00:00 Downloading s3transfer-0.7.0-py3-none-any.whl (79 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.8/79.8 kB 8.6 MB/s eta 0:00:00 Downloading urllib3-2.0.7-py3-none-any.whl (124 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 kB 23.1 MB/s eta 0:00:00 Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 Successfully installed boto3-1.28.77 botocore-1.31.77 jmespath-1.0.1 python-dateutil-2.8.2 s3transfer-0.7.0 six-1.16.0 urllib3-2.0.7 [notice] A new release of pip is available: 23.2.1 -> 23.3.1 [notice] To update, run: pip install --upgrade pip SAM CLI now collects telemetry to better understand customer needs. You can OPT OUT and disable telemetry collection by setting the environment variable SAM_CLITELEMETRY=0 in your shell. Thanks for your help! Learn More: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-telemetry.html Building codeuri: /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/provisioning-operations-handler runtime: python3.9 metadata: {} architecture: x8664 functions: TerraformEngineProvisioningHandlerLambda, TerraformEngineTerminateHandlerLambda Running PythonPipBuilder:ResolveDependencies Running PythonPipBuilder:CopySource Building codeuri: /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/terraform_open_source_parameter_parser runtime: go1.x metadata: {} architecture: x86_64 functions: ParameterParser, ExternalParameterParser Running GoModulesBuilder:Build Build Failed Error: GoModulesBuilder:Build - Builder Failed: stat /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/terraform_open_source_parameter_parser/terraform_open_source_parameter_parser: directory not found Cleaning up project directory and file based variables 00:00 ERROR: Job failed: exit code 1

KINDLY HELP.

kaul-abhi commented 8 months ago

base docker image is of alpine and packages installed are : bash \ py-pip \ jq \ figlet \ go \ wget \ unzip \ git \ gawk \ build-base \ python3 \ py3-pip \ zlib-dev \ readline-dev \ ncurses-dev \ openssl-dev \ sqlite-dev \ tk-dev \ gdbm-dev \ libc-dev \ bzip2 \

Required for testing with go

gcompat \
libc-dev \
zip \
gcc \
# Required for information layer serverless apps local development
make \
openssl \
# Used with aws-cli
curl \
groff \
kaul-abhi commented 8 months ago

@zillidan @smaly-amazon this error had come before also. https://github.com/aws-samples/service-catalog-engine-for-terraform-os/issues/27. Please help