Failing Mounting call_event_processor

Geoffmoss77 commented 5 days ago

Describe the bug docker build of Fetching public.ecr.aws/sam/build-python3.11:latest-arm64 Docker container image................................................................................................................................................................................................................................................................................................................................................................................................................................... Mounting /home/ec2-user/environment/aws/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/call_event_processor as /tmp/samcli/source:ro,delegated, inside runtime container

stuck for hours

To Reproduce Steps to reproduce the behavior:

EC9 fresh install with 40 GB Drive
./publish.sh lca-deploy-bob lca-2309-bob eu-west-2

Expected behavior build job for link to AWS and template file

Screenshots

Skipping building resources inside these nested stacks.                                                                                                                                                                              
Valid cache found, copying previously built resources for following layers (TranscriptEnrichmentPythonLayer)                                                                                                                         
Valid cache found, copying previously built resources for following functions (AsyncTranscriptSummaryOrchestrator)                                                                                                                   
Valid cache found, copying previously built resources for following functions (LLMAnthropicSummaryLambda)                                                                                                                            
Valid cache found, copying previously built resources for following functions (BedrockSummaryLambda)                                                                                                                                 
Valid cache found, copying previously built resources for following functions (AsyncAgentAssistOrchestrator)                                                                                                                         
Valid cache found, copying previously built resources for following functions (LambdaCodeBuildStartBuild)                                                                                                                            
Valid cache found, copying previously built resources for following functions (FetchTranscript)                                                                                                                                      
Cache is invalid, running build and copying resources for following functions (CallEventProcessorFunction)                                                                                                                           
Building codeuri: /home/ec2-user/environment/aws/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/call_event_processor runtime: python3.11 metadata: {'cfn_nag': {'rules_to_suppress': [{'id': 'W89',      
'reason': 'Customer can use VPC if desired'}]}} architecture: arm64 functions: CallEventProcessorFunction                                                                                                                            

Fetching public.ecr.aws/sam/build-python3.11:latest-arm64 Docker container image...................................................................................................................................................................................................................................................................................................................................................................................................................................
Mounting /home/ec2-user/environment/aws/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/call_event_processor as /tmp/samcli/source:ro,delegated, inside runtime container

Sep 23 16:17:43 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:17:43.980326345Z" level=info msg="[graphdriver] using prior storage driver: overlay2"
Sep 23 16:17:43 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:17:43.982962556Z" level=info msg="Loading containers: start."
Sep 23 16:17:44 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:17:44.954064022Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Sep 23 16:17:45 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:17:45.050955517Z" level=info msg="Loading containers: done."
Sep 23 16:17:45 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:17:45.094152440Z" level=info msg="Docker daemon" commit=b08a51f containerd-snapshotter=false storage-driver=overlay2 version=25.0.6
Sep 23 16:17:45 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:17:45.094239751Z" level=info msg="Daemon has completed initialization"
Sep 23 16:17:45 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:17:45.127567494Z" level=info msg="API listen on /run/docker.sock"
Sep 23 16:17:45 ip-172-31-0-136.eu-west-2.compute.internal systemd[1]: Started docker.service - Docker Application Container Engine.
Sep 23 16:21:37 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:21:37.290934731Z" level=warning msg="failed to prune image docker.io/multiarch/qemu-user-static@sha256:fe60359c92e86a43cc87b3d906006245f77bfc0565676b80004cc666e4feb9f0: No such image: multiarch/qemu-user-static@sha256:fe60359c92e86a43cc87b3d906006245f77bfc0565676b80004cc666e4feb9f0"
Sep 23 16:21:39 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:21:39.428384068Z" level=warning msg="failed to prune image public.ecr.aws/sam/build-python3.11@sha256:5efdf71aeb78b083065d2fe054035c93c86ccc011931b06dee2e608a1f91bc18: No such image: public.ecr.aws/sam/build-python3.11@sha256:5efdf71aeb78b083065d2fe054035c93c86ccc011931b06dee2e608a1f91bc18"
Sep 23 16:27:23 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:27:23.194732728Z" level=info msg="ignoring event" container=fe6ece943d7e30001b2fa30b55dc1c395866e78053fd2903c3e8362667ba0679 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Sep 23 16:28:05 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: time="2024-09-23T16:28:05.154805394Z" level=info msg="ignoring event" container=99e9dfa59513c536ada6afa78d049d969f0d6167d44a40e8d387f0e61cf7b5db module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Sep 23 16:30:47 ip-172-31-0-136.eu-west-2.compute.internal dockerd[79091]: 2024/09/23 16:30:47 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)

Additional context Add any other context about the problem here.

Geoffmoss77 commented 4 days ago

we manually pulled the image with : docker pull public.ecr.aws/sam/build-python3.11:latest-arm64

rstrahan commented 4 days ago

Hi @Geoffmoss77 Thanks for reporting and closing.. Glad you got it resolved. I'm unclear though, if there is a remaining defect.. you should not have needed to manually pull the image. Or was it a transient problem, do you think?

Geoffmoss77 commented 4 days ago

Hi Bob

Thanks for the reply, we are not sure. It just hangs, we've left it for hours, We are using cloud 9 instances for building.

I know you should not add 2 issues together, but we found the only way to deploy from cloud9 is using these commands, we tried multiple instances using AWS Linux, and switched to Ubuntu with the following.

sudo pip install virtualenv sudo apt remove python3-blinker sudo pip install aws-sam-cli --force cd amazon-transcribe-live-call-analytics/ ./publish.sh lca-ubt-249 lca-ubt-prfx-249 eu-west-2

After multiple failed builds we finally found that we were getting a 403 on https://s3.eu-west-2.amazonaws.com/lca-deploy-eu-west-2/lca-2309/0.9.2/lca-ai-stack/0.9.2/template.yaml

We stripped the Bot section from the code to bypass the stuck image pull, and we put Docker in debug mode but still did not get any errors :(

Thanks again for any assistance.

rstrahan commented 4 days ago

Thanks Geoff.. I'll try it again from scratch myself with a brand new EC2 instance. Will report back.

rstrahan commented 4 days ago

Not able to reproduce it, unfortunately (or fortunately) @Geoffmoss77 Here are my notes::

Launch new EC2 instance in console.. Config

OS: Amazon Linux 2023
Instance type: t2.large
Storage: 100Gb
Default VPC

Setup

# developer tools & python/pip
sudo yum groupinstall "Development Tools"
sudo yum install python3
sudo yum install python3-pip
pip3 install virtualenv

# docker
sudo yum install docker
sudo service docker start
sudo usermod -a -G docker ec2-user
sudo usermod -a -G docker ssm-user

Log out and back in to apply docker changes

# sam cli
wget https://github.com/aws/aws-sam-cli/releases/latest/download/aws-sam-cli-linux-x86_64.zip
unzip aws-sam-cli-linux-x86_64.zip -d ./sam-cli
sudo ./sam-cli/install

# node 18
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
nvm install 18

AWS CLI credentials

aws configure

Clone LCA repo

git clone https://github.com/aws-samples/amazon-transcribe-live-call-analytics.git

Run LCA publish script

cd amazon-transcribe-live-call-analytics/
./publish.sh bobs-test-lca-build testlca us-east-1

It all ran fine.. final output

Validating template: https://s3.us-east-1.amazonaws.com/bobs-test-lca-build-us-east-1/testlca/lca-main.yaml
OUTPUTS
Template URL: https://s3.us-east-1.amazonaws.com/bobs-test-lca-build-us-east-1/testlca/lca-main.yaml
CF Launch URL: https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://s3.us-east-1.amazonaws.com/bobs-test-lca-build-us-east-1/testlca/lca-main.yaml&stackName=LCA
CLI Deploy: aws cloudformation deploy --region us-east-1 --template-file /tmp/lca/lca-main.yaml --capabilities CAPABILITY_NAMED_IAM CAPABILITY_AUTO_EXPAND --stack-name LCA --parameter-overrides AdminEmail=jdoe@example.com CallAudioSource=Demo Asterisk PBX Server demoSoftphoneAllowedCidr=CIDRBLOCK siprecAllowedCidrList="" S3BucketName=""
Done

rstrahan commented 4 days ago

I claimed victory too soon.. Although the publish script did not hang, it appears it also failed to create the template for the nested AISTACK.. Digging..

Geoffmoss77 commented 4 days ago

I've started another Cloud 9 instance, let us know when you want me to test again :)

Also thanks

rstrahan commented 4 days ago

Different symptom and different root cause though..

PACKAGING lca-ai-stack
~/amazon-transcribe-live-call-analytics/lca-ai-stack/deployment ~/amazon-transcribe-live-call-analytics
make: bash: No such file or directory
make: *** [Makefile:39: out] Error 127
echo '[INFO] uploading sagemaker-summary-stack.yaml to s3://bobs-test-lca-build-us-east-1/testlca/0.9.2/lca-ai-stack/0.9.2/sagemaker-summary-stack.yaml'

When running make in lca-ai-stack it's not finding the bash command..

Tracked it down to this line in the Makefile.. which works on my AL2 Cloud9 build machine, but on the new AL 2023 instance it is effectively messing up the PATH. I'll continue digging.

rstrahan commented 4 days ago

It looks like the newer version of gnu make does not like the " characters in this line

I modified it from

export PATH := "$(VIRTUALENV_BUILD_BIN_DIR):$(VIRTUALENV_DEV_BIN_DIR):$(PATH)"

to

export PATH := $(VIRTUALENV_BUILD_BIN_DIR):$(VIRTUALENV_DEV_BIN_DIR):$(PATH)

which resolved the issue.. Now the publish script ran completely, this time correctly building the AISTACK template and artifacts as well.. No issues with sam containers.

I have a stack deploying currently to verify the build was all good, and i can see it's already deploying the nested AISTACK.. so we definately got past the point where you had been having the problem.

Here's the extract from the publish log for the sam part..

Starting Build use cache
Starting Build inside a container
Below nested stacks(s) specify non-local URL(s), which are unsupported:
- SageMakerTranscriptSummaryStack
- BedrockPreviewBoto3Stack
Skipping building resources inside these nested stacks.
Cache is invalid, running build and copying resources for following layers (TranscriptEnrichmentPythonLayer)
Building layer 'TranscriptEnrichmentPythonLayer'
WARNING: Layer 'TranscriptEnrichmentPythonLayer' has BuildArchitecture x86_64, which is not listed in CompatibleArchitectures

Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image......
Mounting /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_layers/transcript_enrichment_layer as /tmp/samcli/source:ro,delegated, inside runtime container
 Running PythonPipBuilder:ResolveDependencies
 Running PythonPipBuilder:CopySource
Cache is invalid, running build and copying resources for following functions (FetchTranscript)
Cache is invalid, running build and copying resources for following functions (LambdaCodeBuildStartBuild)
Building codeuri: /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/start_codebuild runtime: python3.11 architecture: x86_64 functions: LambdaCodeBuildStartBuild
Building codeuri: /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/fetch_transcript runtime: python3.11 architecture: x86_64 functions: FetchTranscript
Cache is invalid, running build and copying resources for following functions (LLMAnthropicSummaryLambda)
Cache is invalid, running build and copying resources for following functions (AsyncAgentAssistOrchestrator)
Building codeuri: /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/async_agent_assist_orchestrator runtime: python3.11 architecture: x86_64 functions: AsyncAgentAssistOrchestrator
Building codeuri: /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/llm_anthropic_summary_lambda runtime: python3.11 architecture: x86_64 functions: LLMAnthropicSummaryLambda
Cache is invalid, running build and copying resources for following functions (AsyncTranscriptSummaryOrchestrator)
Building codeuri: /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/async_transcript_summary_orchestrator runtime: python3.11 architecture: x86_64 functions: AsyncTranscriptSummaryOrchestrator
Cache is invalid, running build and copying resources for following functions (BedrockSummaryLambda)
Building codeuri: /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/bedrock_summary_lambda runtime: python3.11 architecture: x86_64 functions: BedrockSummaryLambda

Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image......
Mounting /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/start_codebuild as /tmp/samcli/source:ro,delegated, inside runtime container

Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image......
Mounting /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/fetch_transcript as /tmp/samcli/source:ro,delegated, inside runtime container

Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image......
Mounting /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/async_agent_assist_orchestrator as /tmp/samcli/source:ro,delegated, inside runtime container

Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image......
Mounting /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/llm_anthropic_summary_lambda as /tmp/samcli/source:ro,delegated, inside runtime container

Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image......
Mounting /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/bedrock_summary_lambda as /tmp/samcli/source:ro,delegated, inside runtime container

Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image......
Mounting /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/async_transcript_summary_orchestrator as /tmp/samcli/source:ro,delegated, inside runtime container
 Running PythonPipBuilder:ResolveDependencies
requirements.txt file not found. Continuing the build without dependencies.
 Running PythonPipBuilder:CopySource
requirements.txt file not found. Continuing the build without dependencies.
 Running PythonPipBuilder:CopySource
Cache is invalid, running build and copying resources for following functions (CallEventProcessorFunction)
Building codeuri: /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/call_event_processor runtime: python3.11 architecture: arm64 functions: CallEventProcessorFunction
 Running PythonPipBuilder:ResolveDependencies
requirements.txt file not found. Continuing the build without dependencies.
 Running PythonPipBuilder:CopySource
 Running PythonPipBuilder:ResolveDependencies

Fetching public.ecr.aws/sam/build-python3.11:latest-arm64 Docker container image.......................................... Running PythonPipBuilder:CopySource
.................................................................. Running PythonPipBuilder:CopySource
... Running PythonPipBuilder:CopySource
................................................................................................................................................................................................................................................................................................................................................................................................................
Mounting /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/call_event_processor as /tmp/samcli/source:ro,delegated, inside runtime container
 Running PythonPipBuilder:ResolveDependencies
 Running PythonPipBuilder:CopySource

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

rstrahan commented 4 days ago

I will log two separate issues.. (1) Make fails to find bash (with new version of gnu make (4.3) #190 (2) Publish script fails to detect make failure in lca-ai-stack during build.. #191

rstrahan commented 4 days ago

For now, can you try again, @Geoffmoss77 ? Try doing it exactly as i did it above, but edit the file lca-ai-stack/Makefile line 130 first, exactly as shown above. Make sure you provision your instance with at least 100Gb storage - in case the hanging issue is related to insufficient space.. (I have a feeling we encountered something like that before)

rstrahan commented 4 days ago

I just pushed PR #192 to develop to address the incidental issues logged above (#190 and #191) @Geoffmoss77 Feel free to checkout develop branch from the repo to do your build test..

Geoffmoss77 commented 3 days ago

Thank you Bob will deploy and report, Thanks again

Geoffmoss77 commented 3 days ago

Hi Bob I ran : ./publish.sh lca-ubt-249-bobtest lca-ubt-prfx-249-bobtest eu-west-2 from a fresh pull


Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image................................................................................................................................................................................................................................................................................................................
Mounting /home/ec2-user/environment/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_layers/transcript_enrichment_layer as /tmp/samcli/source:ro,delegated, inside runtime container                                                                         

Error: Container does not exist. Cannot get logs for this container
Traceback:
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "samcli/cli/cli_config_file.py", line 347, in wrapper
  File "click/decorators.py", line 92, in new_func
  File "click/core.py", line 783, in invoke
  File "samcli/lib/telemetry/metric.py", line 185, in wrapped
  File "samcli/lib/telemetry/metric.py", line 150, in wrapped
  File "samcli/lib/utils/version_checker.py", line 43, in wrapped
  File "samcli/cli/main.py", line 95, in wrapper
  File "samcli/commands/build/command.py", line 175, in cli
  File "samcli/commands/build/command.py", line 264, in do_cli
  File "samcli/commands/build/build_context.py", line 284, in run
  File "samcli/lib/build/app_builder.py", line 225, in build
  File "samcli/lib/build/build_strategy.py", line 413, in build
  File "samcli/lib/build/build_strategy.py", line 82, in build
  File "samcli/lib/build/build_strategy.py", line 416, in _build_layers
  File "samcli/lib/build/build_strategy.py", line 435, in _run_builds_async
  File "samcli/lib/utils/async_utils.py", line 137, in run_async
  File "samcli/lib/utils/async_utils.py", line 96, in run_given_tasks_async
  File "asyncio/base_events.py", line 653, in run_until_complete
  File "samcli/lib/utils/async_utils.py", line 62, in _run_given_tasks_async
  File "concurrent/futures/thread.py", line 58, in run
  File "samcli/lib/build/build_strategy.py", line 443, in build_single_layer_definition
  File "samcli/lib/build/build_strategy.py", line 608, in build_single_layer_definition
  File "samcli/lib/build/build_strategy.py", line 358, in build_single_layer_definition
  File "samcli/lib/build/build_strategy.py", line 239, in build_single_layer_definition
  File "samcli/lib/build/app_builder.py", line 566, in _build_layer
  File "samcli/lib/build/app_builder.py", line 958, in _build_function_on_container
  File "samcli/local/docker/container.py", line 460, in wait_for_logs

An unexpected error was encountered while executing "sam build".
Search for an existing issue:
https://github.com/aws/aws-sam-cli/issues?q=is%3Aissue+is%3Aopen+Bug%3A%20sam%20build%20-%20RuntimeError
Or create a bug report:
https://github.com/aws/aws-sam-cli/issues/new?template=Bug_report.md&title=Bug%3A%20sam%20build%20-%20RuntimeError
make: *** [Makefile:292: out/template-packaged-lca-ubt-249-bobtest-eu-west-2-lca-ubt-prfx-249-bobtest_0.9.2_lca-ai-stack-0.9.2.yaml] Error 1

Do I need to add another dependency?

rstrahan commented 3 days ago

Hi Geoff, Can you reproduce my successful build by replicating everything exactly as I had noted? In theory that should work, and then we can establish that as a baseline, to figure out what's different in the one that fails. Eg instance size, storage, versions, different region, etc.

On Sep 25, 2024 8:53 AM, Geoffrey Rowland @.***> wrote:

Hi Bob I ran : ./publish.sh lca-ubt-249-bobtest lca-ubt-prfx-249-bobtest eu-west-2 from a fresh pull

`Fetching public.ecr.aws/sam/build-python3.11:latest-x86_64 Docker container image................................................................................................................................................................................................................................................................................................................ Mounting /home/ec2-user/environment/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_layers/transcript_enrichment_layer as /tmp/samcli/source:ro,delegated, inside runtime container

Error: Container does not exist. Cannot get logs for this container Traceback: File "click/core.py", line 1078, in main File "click/core.py", line 1688, in invoke File "click/core.py", line 1434, in invoke File "click/core.py", line 783, in invoke File "samcli/cli/cli_config_file.py", line 347, in wrapper File "click/decorators.py", line 92, in new_func File "click/core.py", line 783, in invoke File "samcli/lib/telemetry/metric.py", line 185, in wrapped File "samcli/lib/telemetry/metric.py", line 150, in wrapped File "samcli/lib/utils/version_checker.py", line 43, in wrapped File "samcli/cli/main.py", line 95, in wrapper File "samcli/commands/build/command.py", line 175, in cli File "samcli/commands/build/command.py", line 264, in do_cli File "samcli/commands/build/build_context.py", line 284, in run File "samcli/lib/build/app_builder.py", line 225, in build File "samcli/lib/build/build_strategy.py", line 413, in build File "samcli/lib/build/build_strategy.py", line 82, in build File "samcli/lib/build/build_strategy.py", line 416, in _build_layers File "samcli/lib/build/build_strategy.py", line 435, in _run_builds_async File "samcli/lib/utils/async_utils.py", line 137, in run_async File "samcli/lib/utils/async_utils.py", line 96, in run_given_tasks_async File "asyncio/base_events.py", line 653, in run_until_complete File "samcli/lib/utils/async_utils.py", line 62, in _run_given_tasks_async File "concurrent/futures/thread.py", line 58, in run File "samcli/lib/build/build_strategy.py", line 443, in build_single_layer_definition File "samcli/lib/build/build_strategy.py", line 608, in build_single_layer_definition File "samcli/lib/build/build_strategy.py", line 358, in build_single_layer_definition File "samcli/lib/build/build_strategy.py", line 239, in build_single_layer_definition File "samcli/lib/build/app_builder.py", line 566, in _build_layer File "samcli/lib/build/app_builder.py", line 958, in _build_function_on_container File "samcli/local/docker/container.py", line 460, in wait_for_logs

An unexpected error was encountered while executing "sam build". Search for an existing issue: https://github.com/aws/aws-sam-cli/issues?q=is%3Aissue+is%3Aopen+Bug%3A%20sam%20build%20-%20RuntimeError Or create a bug report: https://github.com/aws/aws-sam-cli/issues/new?template=Bug_report.md&title=Bug%3A%20sam%20build%20-%20RuntimeError make: *** [Makefile:292: out/template-packaged-lca-ubt-249-bobtest-eu-west-2-lca-ubt-prfx-249-bobtest_0.9.2_lca-ai-stack-0.9.2.yaml] Error 1 `

Do I need to add another dependency?

— Reply to this email directly, view it on GitHubhttps://github.com/aws-samples/amazon-transcribe-live-call-analytics/issues/189#issuecomment-2374004051, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACTSFHRBH6LIOO4XBWVQE23ZYKW5BAVCNFSM6AAAAABOWPKTX6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZUGAYDIMBVGE. You are receiving this because you commented.Message ID: @.***>

Geoffmoss77 commented 3 days ago

Hi Bob Seems to have got stuck at

will give it a few could this be a docker version issue?

This is what I have:

OS: Amazon Linux 2023 Instance type: t2.large Storage: 100Gb Default VPC

commands run:

sudo -i yum groupinstall "Development Tools" -y yum install python3 -y yum install python3-pip -y pip3 install virtualenv yum install npm yum install docker -y systemctl enable docker systemctl start docker <-- not sure why but the script could not see docker could even though it was running sudo usermod -a -G docker ec2-user sudo usermod -a -G docker ssm-user <-- this user not found git clone https://github.com/aws-samples/amazon-transcribe-live-call-analytics.git imported AWS creds from login screen cd /home/ec2-user/amazon-transcribe-live-call-analytics/ ./publish.sh bob-test-eu-west-2 bob-test-deploy eu-west-2

will try US now

On a corporate network and can't access US servers

rstrahan commented 1 day ago

I wish I could explain this.. Any more luck @Geoffmoss77 ? I'll try one more time myself, repeating the experiment I posted above, previously successful, and see what happens.. @kishd - you have also experienced same/similar issues - please post if you make any breakthrough on root cause. Tx.

rstrahan commented 1 day ago

So... I thought I'd record a screen video of me setting up a new EC2, exactly as I did a couple of days ago, and prove beyond doubt that it works... Well.. this time it didn't.. i followed my own script exactly - and this time it's hung Mounting /home/ssm-user/amazon-transcribe-live-call-analytics/lca-ai-stack/source/lambda_functions/call_event_processor as /tmp/samcli/source:ro,delegated, inside runtime container - same as you!

But - at least now I have reproduced the issue.. and am more motivated than ever to help you get to the bottom of it! It's the weekend now, but @kishd and I will dig deep next week - there has got to be a solution!

Hope you have a great weekend!

aws-samples / amazon-transcribe-live-call-analytics