localstack / serverless-localstack

⚡ Serverless plugin for running against LocalStack
511 stars 82 forks source link

Recoverable error occurred (Inaccessible host: `engine-localstack' at port `undefined'... only when running from within dockerfile script #207

Closed mishabruml closed 6 months ago

mishabruml commented 1 year ago
serverless.yml ```yml localstack: stages: - dev host: http://engine-localstack edgePort: 4566 autostart: false ```
docker-compose.yml ```yml localstack: image: localstack/localstack:1.3.1 hostname: engine-localstack container_name: engine-localstack restart: on-failure environment: - SERVICES=s3,sqs,lambda - DEFAULT_REGION=eu-west-1 - DOCKER_HOST=unix:///var/run/docker.sock - DEBUG=true - LAMBDA_DOCKER_NETWORK=localstack-network - PROVIDER_OVERRIDE_LAMBDA=asf # https://github.com/localstack/localstack/issues/7209 - HOSTNAME=engine-localstack ports: - '4566:4566' - '4571:4571' volumes: - '${TMPDIR:-/tmp/localstack}:/var/lib/localstack' - '/var/run/docker.sock:/var/run/docker.sock' localstack-serverless-deployer: hostname: engine-localstack-serverless-deployer container_name: engine-localstack-serverless-deployer volumes: - /var/run/docker.sock:/var/run/docker.sock build: dockerfile: localstack/Dockerfile.localstack-serverless-deployer depends_on: - localstack ```
Dockerfile.localstack-serverless-deployer ```dockerfile # Container to deploy serverless stack to localhost FROM node:18 AS docker-build WORKDIR /engine COPY package*.json ./ RUN npm install COPY tsconfig.json ./ COPY src ./src COPY serverless.yml serverless.yml RUN npx sls deploy --verbose ```

From a clean docker desktop env, run: docker-compose up -d localstack And then docker compose build localstack-serverless-deployer --progress plain

This yeilds logs: #12 9.870 Recoverable error occurred (Inaccessible host: 'engine-localstack' at port 'undefined'. This service may not be available in the 'eu-west-1' region.), sleeping for ~6 seconds. Try 1 of 4

Full logs ``` #1 [internal] load build definition from Dockerfile.localstack-serverless-deployer #1 transferring dockerfile: 63B done #1 DONE 0.1s #2 [internal] load .dockerignore #2 transferring context: 35B 0.0s done #2 DONE 0.0s #3 [internal] load metadata for docker.io/library/node:18 #3 DONE 1.3s #4 [1/8] FROM docker.io/library/node:18@sha256:d871edd5b68105ebcbfcde3fe8c79d24cbdbb30430d9bd6251c57c56c7bd7646 #4 DONE 0.0s #5 [internal] load build context #5 transferring context: 25.39kB 0.2s done #5 DONE 0.2s #6 [3/8] COPY package*.json ./ #6 CACHED #7 [4/8] RUN npm install #7 CACHED #8 [5/8] COPY tsconfig.json ./ #8 CACHED #9 [6/8] COPY src ./src #9 CACHED #10 [2/8] WORKDIR /engine #10 CACHED #11 [7/8] COPY serverless.yml serverless.yml #11 CACHED #12 [8/8] RUN npx sls deploy --verbose #12 5.908 Using serverless-localstack #12 9.436 #12 9.436 Deploying engine to stage dev (eu-west-1) #12 9.437 #12 9.479 Compiling to node18 bundle with esbuild... #12 9.483 Compiling with concurrency: Infinity #12 9.870 Recoverable error occurred (Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region.), sleeping for ~6 seconds. Try 1 of 4 #12 13.83 Compiling completed. #12 13.84 Warning: No external modules needed #12 16.40 Recoverable error occurred (Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region.), sleeping for ~5 seconds. Try 2 of 4 #12 22.96 Recoverable error occurred (Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region.), sleeping for ~5 seconds. Try 3 of 4 #12 23.61 Zip function: run-db-migrations - 11.19 MB [1766 ms] #12 23.62 Zip function: test - 11.19 MB [4322 ms] #12 24.18 Recoverable error occurred (Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region.), sleeping for ~6 seconds. Try 1 of 4 #12 28.48 Recoverable error occurred (Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region.), sleeping for ~4 seconds. Try 4 of 4 #12 30.14 Recoverable error occurred (Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region.), sleeping for ~5 seconds. Try 2 of 4 #12 35.64 Recoverable error occurred (Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region.), sleeping for ~6 seconds. Try 3 of 4 #12 42.63 Recoverable error occurred (Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region.), sleeping for ~4 seconds. Try 4 of 4 #12 47.23 #12 47.23 × Stack engine-dev failed to deploy (37s) #12 47.23 Environment: linux, node 18.13.0, framework 3.27.0 (local), plugin 6.2.3, SDK 4.3.2 #12 47.23 Credentials: Local, environment variables #12 47.23 Docs: docs.serverless.com #12 47.23 Support: forum.serverless.com #12 47.23 Bugs: github.com/serverless/serverless/issues #12 47.23 #12 47.23 Error: #12 47.23 Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region. #12 ERROR: executor failed running [/bin/sh -c npx sls deploy --verbose]: exit code: 1 ------ > [8/8] RUN npx sls deploy --verbose: #12 47.23 #12 47.23 × Stack engine-dev failed to deploy (37s) #12 47.23 Environment: linux, node 18.13.0, framework 3.27.0 (local), plugin 6.2.3, SDK 4.3.2 #12 47.23 Credentials: Local, environment variables #12 47.23 Docs: docs.serverless.com #12 47.23 Support: forum.serverless.com #12 47.23 Bugs: github.com/serverless/serverless/issues #12 47.23 #12 47.23 Error: #12 47.23 Inaccessible host: `engine-localstack' at port `undefined'. This service may not be available in the `eu-west-1' region. ------ failed to solve: executor failed running [/bin/sh -c npx sls deploy --verbose]: exit code: 1 ```

What is super interesting/annoying about this issue is that with the exact same setup, running docker compose run localstack-serverless-deployer npx sls deploy --verbose works!

Full logs ``` $ docker compose run localstack-serverless-deployer npx sls deploy --verbose [+] Running 1/0 ⠿ Container engine-localstack Running 0.0s Using serverless-localstack Deploying engine to stage dev (eu-west-1) Compiling to node18 bundle with esbuild... Compiling with concurrency: Infinity Compiling completed. Warning: No external modules needed Zip function: run-db-migrations - 11.19 MB [1307 ms] Zip function: test - 11.19 MB [3454 ms] Creating new change set Waiting for new change set to be created Executing created change set CREATE_IN_PROGRESS - AWS::CloudFormation::Stack - engine-dev CREATE_COMPLETE - AWS::CloudFormation::Stack - ServerlessDeploymentBucket CREATE_COMPLETE - AWS::CloudFormation::Stack - ServerlessDeploymentBucketPolicy CREATE_COMPLETE - AWS::CloudFormation::Stack - engine-dev Uploading CloudFormation file to S3 Uploading State file to S3 Uploading service test.zip file to S3 (11.73 MB) Uploading service run-db-migrations.zip file to S3 (11.74 MB) Skipping template validation: Unsupported in Localstack Creating new change set Waiting for new change set to be created Executing created change set UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - ServerlessDeploymentBucketPolicy CREATE_COMPLETE - AWS::CloudFormation::Stack - TestLogGroup CREATE_COMPLETE - AWS::CloudFormation::Stack - RunDashdbDashmigrationsLogGroup CREATE_COMPLETE - AWS::CloudFormation::Stack - IamRoleLambdaExecution CREATE_COMPLETE - AWS::CloudFormation::Stack - TestLambdaFunction CREATE_COMPLETE - AWS::CloudFormation::Stack - RunDashdbDashmigrationsLambdaFunction CREATE_COMPLETE - AWS::CloudFormation::Stack - TestLambdaVersionWY5FGxphepHypRgXaj3G8Qwr3LyYTwGwS0Tvgpn0 CREATE_COMPLETE - AWS::CloudFormation::Stack - RunDashdbDashmigrationsLambdaVersionoh86AQf7Yu64pCOHL0pgFYErkvztrfDIhVccbDND3ww UPDATE_COMPLETE - AWS::CloudFormation::Stack - ServerlessDeploymentBucket UPDATE_COMPLETE - AWS::CloudFormation::Stack - ServerlessDeploymentBucketPolicy UPDATE_COMPLETE - AWS::CloudFormation::Stack - engine-dev Removing old service artifacts from S3 ✔ Service deployed to stack engine-dev (30s) Need a better logging experience than CloudWatch? Try our Dev Mode in console: run "serverless --console" ```
mishabruml commented 1 year ago

Update:

changing the localstack host config in serverless.yml to host: http://host.docker.internal allows the npx sls deploy command to succeed in the context of a docker build script.

This solution isn't desired though as host.docker.internal can't be resolved on all systems, and isn't designed for use outside of docker desktop. We are looking to run these deployments in CI as part of test suite

whummer commented 1 year ago

Thanks for reporting @mishabruml . Have you been able to verify if there is connectivity from the localstack-serverless-deployer container to the engine-localstack host on port 4566? You could exec into the container and give it a try, e.g., by running a simple curl engine-localstack:4566 command. Hope that helps - please keep us posted on how it goes.. Thanks!

steffyP commented 8 months ago

Hi @mishabruml,

could you provide an update on your issue? Were you able to verify the connectivity to the container?

steffyP commented 6 months ago

This issue seems to be stale. As we didn't get any response in two months, we are closing this issue now. Please feel free to reopen if it still persists, and provide updates logs and details.