aws-solutions / clickstream-analytics-on-aws

Build clickstream analytics on AWS for your mobile and web applications
https://aws.amazon.com/solutions/implementations/clickstream-analytics-on-aws/
Apache License 2.0
71 stars 22 forks source link

Can't re-deploy stack from deleted stack ARN #1567

Closed AdamUrbanfox closed 2 months ago

AdamUrbanfox commented 3 months ago

:question: When running the following command:

bash e2e-deploy.sh -n ingestToKafkaStackName -s stack_arn -p profile

The follwoing errors occur relating to package resolution:

[ERROR] Could not resolve "@aws/clickstream-base-lib"

src/control-plane/backend/lambda/batch-insert-ddb/index.ts:14:89:
  14 │ import { aws_sdk_client_common_config, logger, marshallOptions, unmarshallOptions } from '@aws/clickstream-base-lib';
     ╵                                                                                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~

The Yarn Plug'n'Play manifest forbids importing "@aws/clickstream-base-lib" here because it's not listed as a dependency of this package:

../../.pnp.cjs:37:31:
  37 │         "packageDependencies": [\
     ╵                                ~~

You can mark the path "@aws/clickstream-base-lib" as external to exclude it from the bundle, which will remove this error and leave the unresolved path in the bundle.

✘ [ERROR] Could not resolve "@aws-sdk/client-dynamodb"

src/control-plane/backend/lambda/batch-insert-ddb/index.ts:19:7:
  19 │ } from '@aws-sdk/client-dynamodb';
     ╵        ~~~~~~~~~~~~~~~~~~~~~~~~~~

The Yarn Plug'n'Play manifest forbids importing "@aws-sdk/client-dynamodb" here because it's not listed as a dependency of this package:

../../.pnp.cjs:37:31:
  37 │         "packageDependencies": [\
     ╵                                ~~

You can mark the path "@aws-sdk/client-dynamodb" as external to exclude it from the bundle, which will remove this error and leave the unresolved path in the bundle.

✘ [ERROR] Could not resolve "@aws-sdk/lib-dynamodb"

src/control-plane/backend/lambda/batch-insert-ddb/index.ts:25:7:
  25 │ } from '@aws-sdk/lib-dynamodb';
     ╵        ~~~~~~~~~~~~~~~~~~~~~~~

The Yarn Plug'n'Play manifest forbids importing "@aws-sdk/lib-dynamodb" here because it's not listed as a dependency of this package:

../../.pnp.cjs:37:31:
  37 │         "packageDependencies": [\
     ╵                                ~~

You can mark the path "@aws-sdk/lib-dynamodb" as external to exclude it from the bundle, which will remove this error and leave the unresolved path in the bundle.

✘ [ERROR] Could not resolve "@aws-sdk/util-dynamodb"

src/control-plane/backend/lambda/batch-insert-ddb/index.ts:26:47:
  26 │ import { NativeAttributeValue, marshall } from '@aws-sdk/util-dynamodb';
     ╵                                                ~~~~~~~~~~~~~~~~~~~~~~~~

The Yarn Plug'n'Play manifest forbids importing "@aws-sdk/util-dynamodb" here because it's not listed as a dependency of this package:

../../.pnp.cjs:37:31:
  37 │         "packageDependencies": [\
     ╵                                ~~

You can mark the path "@aws-sdk/util-dynamodb" as external to exclude it from the bundle, which will remove this error and leave the unresolved path in the bundle.

The Question

Other information

I have deleted node_modules and the lock file and re-run pnpm install && pnpm projen && pnpm nx build @aws/clickstream-base-lib with no luck.

zxkane commented 3 months ago

@AdamUrbanfox Did you run the script against the main branch?

Could you please restore the pnpm-lock.yaml(it makes sure the dependencies have certain versions validated by CI build) from source control and then rerun the command to build the shared module @aws/clickstream-base-lib before deploying the stacks from source code?

AdamUrbanfox commented 3 months ago

I tried doing this on a fresh repo, with the pnpm-lock.yaml file restored directly from the github repo. I encounter the same error

I have already run the pnpm command which seems to have been successful?

`% pnpm install && pnpm projen && pnpm nx build @aws/clickstream-base-lib Scope: all 5 workspace projects Lockfile is up to date, resolution step is skipped Already up to date Done in 1s

clickstream-analytics-on-aws@1.2.0 projen /Users/adamprior/vscode/urbanfox-clickstream-pipeline pnpm dlx projen

Packages: +2 ++ Progress: resolved 2, reused 2, downloaded 0, added 2, done 👾 default | node .projenrc.js

nx run @aws/clickstream-base-lib:build [existing outputs match the cache, left as is]

@aws/clickstream-base-lib@1.2.0 build /Users/adamprior/vscode/urbanfox-clickstream-pipeline/src/base-lib pnpm dlx projen build

Progress: resolved 1, reused 0, downloaded 0, added 0 Packages: +2 ++ Progress: resolved 2, reused 2, downloaded 0, added 2, done 👾 build » compile tsc --build 👾 build » test jest --passWithNoTests --updateSnapshot PASS test/solution-info.test.ts PASS test/utils.test.ts
File % Stmts % Branch % Funcs % Lines Uncovered Line #s
------------------- --------- ---------- --------- --------- --------------------------------------
All files 94.32 92.98 77.77 94.32
common 80.51 87.87 77.77 80.51
powertools.ts 100 100 100 100
solution-info.ts 96.93 86.2 100 96.93 62,70-71,187-189
utils.ts 49.59 100 40 49.59 20-28,30-51,53-55,81-87,89-93,95-110
constant 100 100 100 100
constant.ts 100 100 100 100
dashboard.ts 100 100 100 100
explore-types.ts 100 100 100 100
index.ts 100 100 100 100
------------------- --------- ---------- --------- --------- --------------------------------------

Test Suites: 2 passed, 2 total Tests: 6 passed, 6 total Snapshots: 0 total Time: 1.998 s Ran all test suites. 👾 build » package | mkdir -p dist/js 👾 build » package | pnpm pack --pack-destination dist/js /Users/adamprior/vscode/urbanfox-clickstream-pipeline/src/base-lib/dist/js/aws-clickstream-base-lib-1.2.0.tgz

————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

NX Successfully ran target build for project @aws/clickstream-base-lib (53ms)

Nx read the output from the cache instead of running the command for 1 out of 1 tasks.`

But I'm still getting errors like:

`✘ [ERROR] Could not resolve "@aws/clickstream-base-lib"

src/metrics/custom-resource/set-metrics-widgets/index.ts:14:53:
  14 │ import { aws_sdk_client_common_config, logger } from '@aws/clickstream-base-lib';
     ╵                                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~

The Yarn Plug'n'Play manifest forbids importing "@aws/clickstream-base-lib" here because it's not listed as a dependency of this package:

../.pnp.cjs:37:31:
  37 │         "packageDependencies": [\
     ╵                                ~~

You can mark the path "@aws/clickstream-base-lib" as external to exclude it from the bundle, which will remove this error and leave the unresolved path in the bundle.`

zxkane commented 3 months ago

@AdamUrbanfox Which OS did you use? The project does not use yarn to manage the dependencies of nodejs. And all Lambda functions are compiled and bundled in the Docker image.

It works for me when deploying from both macOS and Ubuntu 22.04 Linux.

AdamUrbanfox commented 3 months ago

I'm using MacOS. node version 22.6 (originally was trying with node 21 so I upgraded to 22.6 with the same error), npm 10.8.2. (originally 10.5, same error).

Again I tried freshly downloading the repo, and even tried pnpm install && pnpm projen && pnpm nx build @aws/clickstream-base-lib through a terminal to make sure it wasn't my IDE affecting things, but still the error occurs.

zxkane commented 3 months ago

Could you upload the complete console output for debugging? I can't reproduce it with the same Node (22.6) and NPM (10.8.2) on macOS 14.6.1.

zxkane commented 2 months ago

Adam, let's know if you have more questions.