Open Analect opened 6 months ago
Hi @Analect, thank you for reaching me out!
I read your log and found that dkStack/LocalBuild/DeployWebsite
is for the 1st option and shouldn't be there if you commented out it. Can you check if it is properly commented out?
The error itself is an network error, so it should success with a few retry, but it ultimately depends on your network environment.
As for the error on cdk diff, I haven't seen that, but can you try the above fix first? I guess it's just a temporary error.
Thanks @tmokmss ... you were right, I hadn't properly commented-out things. Sorry!
I had a quick follow-on question, if you can help clarify, that would be much appreciated.
On the backend, I can see you assign values to the NEXT_PUBLIC_*
env variables here.
An then these are consumed on the front-end, per below.
const amplifyConfig = {
Auth: {
region: process.env.NEXT_PUBLIC_AWS_REGION!,
userPoolId: process.env.NEXT_PUBLIC_USER_POOL_ID!,
userPoolWebClientId: process.env.NEXT_PUBLIC_USER_POOL_CLIENT_ID!,
},
};
I am looking to do something similar with React and noticed your aws-samples/serverless-full-strack-webapp-starter-kit
is setup using React on the front-end, leveraging amplify v5.
If I look here, I can see assignment of values to VITE_*
env variables.
buildEnvironment: {
VITE_BACKEND_API_URL: props.backendApi.apiEndpoint,
VITE_USER_POOL_ID: props.auth.userPool.userPoolId,
VITE_USER_POOL_CLIENT_ID: props.auth.client.userPoolClientId,
VITE_AWS_REGION: Stack.of(props.auth.userPool).region,
Then config.js
is somehow used as an intermediary using import.meta.env
syntax.
export default {
apiEndpoint: import.meta.env.VITE_BACKEND_API_URL!,
userPoolId: import.meta.env.VITE_USER_POOL_ID!,
awsRegion: import.meta.env.VITE_AWS_REGION!,
userPoolClientId: import.meta.env.VITE_USER_POOL_CLIENT_ID!,
};
... which is then consumed here
{
Auth: {
region: config.awsRegion,
userPoolId: config.userPoolId,
userPoolWebClientId: config.userPoolClientId,
},
There seems to be a fair bit of confusion around how this would all work with v6 of amplify, per this issue. I was wondering if you have implemented anything with amplify v6 and React?
One of those on that issue thread ended up having to come up with a more involved solution ... admittedly with NextJS. I'm just hoping the React-based approach is easier.
Thanks alot.
I haven't used Amplify v6, but as far as it uses public environment variables, it should work in the same way.
Regarding v5 to v6 migration, it seems you can just change the schema of Amplify config like this:
// v5
Amplify.configure({
Auth: {
userPoolId: process.env.REACT_APP_USERPOOLID,
userPoolWebClientId: process.env.REACT_APP_USERPOOLWEBCLIENTID,
identityPoolId: process.env.REACT_APP_IDPOOLID,
region: process.env.REACT_APP_REGION
},
});
// v6
Amplify.configure({
Auth: {
Cognito: {
userPoolId: process.env.REACT_APP_USERPOOLID,
userPoolClientId: process.env.REACT_APP_USERPOOLWEBCLIENTID,
identityPoolId: process.env.REACT_APP_IDPOOLID
}
},
Ah OK. I'll give it a try and report back. Have a good evening.
Yeah let me know if you stumble upon something π
@tmokmss ... I have been experimenting with your deploy-time-build
construct, trying to get it to run in the context of a repo generated with projen
. I had previously tested deploying this repo deploy-static-web-frontend-with-cdk
, which worked fine.
I know that projen is more opinionated in terms of its tooling and I experienced a problem with the build failing on deploy, which ends up reverting the whole stack-building process ... one of the negative consequences of going the deploy-time-build
route.
I wanted to get to the bottom what I might be doing wrong.
For context, here is a high-level view of the projen-cdk-serverless-rag
repo I'm experimenting with, which I built-out using projen
. It's missing the various hidden files and directories like .projen
etc.., but I don't think that matters for what I describe here. The infrastructure built-out by CDK (including the deploy-time-build
construct) is at src/main.ts
, while the front-end app is at frontend/src/App.tsx
.
.
βββ cdkCFExecutionPolicy.json
βββ cdk.json
...
βββ frontend
β βββ build
β β βββ static
β β βββ js
β β βββ 488.535652aa.chunk.js
β β βββ 488.535652aa.chunk.js.map
β β βββ main.a7985a79.js
β β βββ main.a7985a79.js.LICENSE.txt
β β βββ main.a7985a79.js.map
β βββ LICENSE
β βββ package.json
β βββ public
β βββ README.md
β βββ src
β β βββ App.css
β β βββ App.test.tsx
β β βββ App.tsx <--- front-end app code is here
β β βββ react-app-env.d.ts
β β βββ reportWebVitals.ts
β β βββ setupTests.ts
β βββ tsconfig.dev.json
β βββ tsconfig.json
β βββ yarn.lock
βββ LICENSE
βββ package.json
βββ README.md
βββ src
β βββ main.ts <---- Where backend infrastructure is defined, including deploy-time-build constructs
βββ test
β βββ main.test.ts
β βββ __snapshots__
β βββ main.test.ts.snap
βββ test-reports
β βββ junit.xml
βββ tsconfig.dev.json
βββ tsconfig.json
βββ yarn.lock
This was the code from src/main.ts
used to describe the app location in the frontend
folder. In your path.join('.', 'frontend'),
in this repo, you had two dots ..
. It wasn't completely clear how that works. Whether you are referencing the location of the frontend
folder from the main.ts
file or from the root of the repo. In any case, I think there are other issues I face too.
new NodejsBuild(parent, 'DeployWebsite', {
assets: [
{
path: path.join('.', 'frontend'),
exclude: readFileSync(path.join('.', 'frontend', '.gitignore')).toString().split('\n'),
ignoreMode: cdk.IgnoreMode.GIT,
},
],
destinationBucket: assetBucket,
distribution,
outputSourceDirectory: 'out',
buildCommands: ['npm ci', 'npm run build'],
buildEnvironment: {
REACT_APP_AWS_REGION: cdk.Stack.of(this).region,
REACT_APP_USER_POOL_ID: userPool.userPoolId,
REACT_APP_USER_POOL_CLIENT_ID: client.userPoolClientId,
REACT_APP_TITLE: 'Deploy Time Build',
},
});
}
In your deploy-static-web-frontend-with-cdk
, it seems the frontend
folder is two levels up from the /cdk/lib/cdk-stack.ts
infrastructure code.
deploy-static-web-frontend-with-cdk/cdk/lib$ ls -ltra ../../frontend
total 776
-rw-rw-r-- 1 xxx 172 Mar 23 12:46 README.md
-rw-rw-r-- 1 xxx 392 Mar 23 12:46 .gitignore
-rw-rw-r-- 1 xxx 40 Mar 23 12:46 .eslintrc.json
-rw-rw-r-- 1 xxx 120 Mar 23 12:46 .env.local.example
-rw-rw-r-- 1 xxx 688 Mar 23 12:46 tsconfig.json
-rw-rw-r-- 1 xxx 712587 Mar 23 12:46 package-lock.json
-rw-rw-r-- 1 xxx 588 Mar 23 12:46 package.json
-rw-rw-r-- 1 xxx 138 Mar 23 12:46 next.config.js
drwxrwxr-x 2 xxx 4096 Mar 23 12:46 app
drwxrwxr-x 4 xxx 4096 Mar 23 12:53 .
drwxr-xr-x 801 root root 36864 Mar 23 13:37 node_modules
drwxrwxr-x 7 xxx 4096 Mar 23 17:17 ..
In my case, for the projen-cdk-serverless-rag
folder, the frontend
colder is one level up from the infra code.
projen-cdk-serverless-rag/src$ ls -ltra ../frontend/src/
total 44
-rw-r--r-- 1 xxx 39 Mar 25 19:36 react-app-env.d.ts
-rw-r--r-- 1 xxx 2671 Mar 25 19:36 logo.svg
-rw-r--r-- 1 xxx 246 Mar 25 19:36 App.test.tsx
-rw-r--r-- 1 xxx 366 Mar 25 19:36 index.css
-rw-r--r-- 1 xxx 564 Mar 25 19:36 App.css
-rw-r--r-- 1 xxx 241 Mar 25 19:36 setupTests.ts
drwxrwxr-x 2 xxx 4096 Mar 25 19:36 .
-rw-r--r-- 1 xxx 501 Mar 25 19:39 index.tsx
-rw-r--r-- 1 xxx 465 Mar 25 19:39 reportWebVitals.ts
drwxrwxr-x 9 xxx 4096 Mar 25 20:36 ..
-rw-r--r-- 1 xxx 1137 Mar 25 22:31 App.tsx
Below are the full logs of the error and below that a snipped of what I think is the relevant part.
The
npm cicommand can only install with an existing package-lock.json or npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm@5 or later to generate a package-lock.json file, then try again.
Per the folder-structure above (and projen preferred tooling), it seems it doesn't produce a package-lock.json
file. In fact, from this issue, it seems projen
was never intended to generate a package-lock.json
file, rather that is output by npm
. But in a projen
context, when running npx projen build
and npx projen deploy
, no such lock-file gets created .... and it seems that that then somehow breaks the deploy-time-build
construct ... unless I'm just not using it properly.
Any thoughts on how I might fix this? Thanks alot.
2024-03-26T11:36:06.228+00:00 drwxr-xr-x 4 root root 133 Mar 26 11:36 frontend
2024-03-26T11:36:06.228+00:00
2024-03-26T11:36:06.228+00:00 [Container] 2024/03/26 11:36:05.787331 Running command ls -la
2024-03-26T11:36:06.228+00:00 total 0
2024-03-26T11:36:06.228+00:00 drwxr-xr-x 3 root root 22 Mar 26 11:36 .
2024-03-26T11:36:06.228+00:00 drwxr-xr-x 3 root root 17 Mar 26 11:35 ..
2024-03-26T11:36:06.228+00:00 drwxr-xr-x 4 root root 133 Mar 26 11:36 frontend
2024-03-26T11:36:06.228+00:00
2024-03-26T11:36:06.228+00:00 [Container] 2024/03/26 11:36:05.793708 Running command cd "$workingDirectory"
2024-03-26T11:36:06.228+00:00
2024-03-26T11:36:06.228+00:00 [Container] 2024/03/26 11:36:05.799251 Running command eval "$buildCommands"
2024-03-26T11:36:18.198+00:00 npm ERR! code EUSAGE
2024-03-26T11:36:18.198+00:00 npm ERR!
2024-03-26T11:36:18.198+00:00 npm ERR! The `npm ci` command can only install with an existing package-lock.json or
2024-03-26T11:36:18.198+00:00 npm ERR! npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm@5 or
2024-03-26T11:36:18.198+00:00 npm ERR! later to generate a package-lock.json file, then try again.
2024-03-26T11:36:18.198+00:00 npm ERR!
2024-03-26T11:36:18.198+00:00 npm ERR! Clean install a project
2024-03-26T11:36:18.198+00:00 npm ERR!
2024-03-26T11:36:18.198+00:00 npm ERR! Usage:
2024-03-26T11:36:18.198+00:00 npm ERR! npm ci
2024-03-26T11:36:18.198+00:00 npm ERR!
2024-03-26T11:36:18.198+00:00 npm ERR! Options:
2024-03-26T11:36:18.198+00:00 npm ERR! [--install-strategy <hoisted|nested|shallow|linked>] [--legacy-bundling]
2024-03-26T11:36:18.198+00:00 npm ERR! [--global-style] [--omit <dev|optional|peer> [--omit <dev|optional|peer> ...]]
2024-03-26T11:36:18.198+00:00 npm ERR! [--strict-peer-deps] [--foreground-scripts] [--ignore-scripts] [--no-audit]
2024-03-26T11:36:18.198+00:00 npm ERR! [--no-bin-links] [--no-fund] [--dry-run]
2024-03-26T11:36:18.198+00:00 npm ERR! [-w|--workspace <workspace-name> [-w|--workspace <workspace-name> ...]]
2024-03-26T11:36:18.198+00:00 npm ERR! [-ws|--workspaces] [--include-workspace-root] [--install-links]
2024-03-26T11:36:18.198+00:00 npm ERR!
2024-03-26T11:36:18.198+00:00 npm ERR! aliases: clean-install, ic, install-clean, isntall-clean
2024-03-26T11:36:18.198+00:00 npm ERR!
2024-03-26T11:36:18.198+00:00 npm ERR! Run "npm help ci" for more info
2024-03-26T11:36:18.198+00:00
2024-03-26T11:36:18.198+00:00 npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-03-26T11_36_13_253Z-debug-0.log
Hi @Analect
It wasn't completely clear how that works. Whether you are referencing the location of the frontend folder from the main.ts file or from the root of the repo
It's based on the current working directory, which is usually where you run cdk command, which is usually the root of the repo. You can also use __dirname
to use relative path from the ts file instead.
I think your current path.join('.', 'frontend'),
configuration is correct.
The npm cicommand can only install with an existing package-lock.json or npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm@5 or later to generate a package-lock.json file, then try again.
If you are using yarn, change the buildCommands
property:
- buildCommands: ['npm ci', 'npm run build'],
+ buildCommands: ['yarn', 'yarn build'],
You might also need to add npm install -g yarn
to make yarn available.
Hope this helps π
Is the outputSourceDirectory: 'out',
setting appropriate?
I presume you wouldn't want to install projen
?
Do you know if the default build-tool is yarn
- I presume so, if there's a yarn.lock
file that has been generated.
I will try switch buildCommands: ['yarn', 'yarn build'],
, as you suggest.
if yarn needs to be installed, should that be added as the first element in that array?
... like buildCommands: ['npm install -g yarn', 'yarn', 'yarn build'],
This is what I run using npx projen build
, which results in:
projen-cdk-serverless-rag/frontend$ npx projen build
πΎ build Β» default | ts-node --project tsconfig.dev.json .projenrc.ts
πΎ build Β» compile | react-scripts build
Is the outputSourceDirectory: 'out', setting appropriate?
outputSourceDirectory
is where your build artifact will be located, so change it according to your configuration (build
I guess?)
if yarn needs to be installed, should that be added as the first element in that array?
Yes. The commands will be executed serially in the array order.
As for projen, I'm not sure what the exact configuration you are using, but if it is required to build your frontend, you should have to install it too.
This construct basically copies your local file to CodeBuild, and run the build command to extract frontend static files. So all you need is 1. include all the necessary files as input, 2. set the required commands to build frontend assets. The commands should be the same as what you run in your local environment.
@tmokmss ... I was interested to listen to your CDK Day presentation and like the idea behind your 3rd option, in terms of leveraging the deploy-time-build construct.
Just to test, I cloned this demo repo locally, ran
npm install
in thecdk
folder, then commented-out this part of the code, as I was most interested in the third approach, then rancdk deploy
. It pulled an image ofnode:18
locally, but then errored-out, as follows:I then tried a
cdk diff
, but that failed too.Is there some pre-requisite setup that I should be doing ... or certain permissionings on AWS beforehand for the authenticated aws user? Thanks.