Closed chrismilleruk closed 2 years ago
@chrismilleruk Could you please use npm install -g @aws-amplify/cli
instead of using yarn and if you're still experiencing this issue? We've seen issues with yarn installations in the past like this one - https://github.com/aws-amplify/amplify-cli/issues/8942
@kaustavghosh06 yes, I switched to yarn after npm i -g @aws-amplify/cli
failed, here is the buildspec and logs from that run:
@chrismilleruk Does it work with the default Node version which Amplify build provides and NPM CLI installation?
It's a reasonable question but it might be trying to create a workaround / resolve a different issue. Specifically (& rather sarcastically), "...upgrade Node AND Amplify prior to amplifyPush in build" definitely does not work if we use the "default Node version which Amplify build provides and NPM CLI installation." π
... however, for your info, my specific use case & background:
In our case, the correct & specific version of tsc
is required for this build step:
The specific tsc
version we need is stored in the package.json file so a simple npm install -g typescript
against node 12
doesn't really work for me as it will introduce inconsistencies.
npm install
with the default 'NPM CLI installation' produces errors due to a package-lock.json
version conflict.
I'm not too keen to look at any temporary workarounds for this issue since a recent opaque change to _LIVE_UPDATES
caused a build pipeline to fail spectacularly, in prod, 24h before a conference.
I've disabled lambda pre-build for now and I'm looking for a robust way to upgrade the build container to the correct versions prior to running the build. Preferably without installing the same thing multiple times against different versions.
I hope this helps & provides the background context you need to understand why I'm attempting to upgrade node
& reinstall amplify
at this point in the build.
@kaustavghosh06 thanks in advance for your help and let me know if I can add any more info for you.
Hi @chrismilleruk, is it possible for you to use node 14
? Please let us know if you're still having issues after that.
Hey @chrismilleruk :wave: as a workaround can you try adding the string "packaged-node-modules"
to the pluginDirectories
section of the ~/.amplify/plugins.json
file and try to redeploy?
{
"pluginDirectories": [
"cli-local-node-modules",
"cli-parent-directory",
"global-node-modules",
+ "packaged-node-modules"
],
// ...
}
For reference: https://github.com/josefaidt/9209/blob/main/scripts/patch.js
@lazpavel, @josefaidt π Apologies for the slow response. I've created a ticket to try these approaches and will let you know how it goes.
Hey @chrismilleruk :wave: just wanted to follow-up on this one and see if you'd had a chance to try out the workaround posted above?
Hi @josefaidt, @lazpavel, I've had a chance to revisit this and test some more this evening, with mixed results. There seem to be a few things overlapping here so I did some brute force testing to try and isolate the key factors.
A quick reminder that the goal (for me at least) is to upgrade node
(Active LTS: v16, Current: v17) with a working amplify
CLI at preBuild
. The two workarounds mentioned are "use node 14" and "add the string "packaged-node-modules" to the pluginDirectories section of the ~/.amplify/plugins.json".
The patch seems to be non-impacting from what I can tell.
The patch works for nvm install v14
and nvm install v16
when using yarn
The target version of Node (and NPM) coupled with the amplify-cli
install method seems to have the greatest impact & correlation.
I see also that the main build provision script has been updated and the default version of node
is now v14.18.1
instead of v12.21.0
. A great improvement π. We need to nvm install
something in order to cause the issue so I have just performed a nvm install 14.18.3
.
NO PATCH
OR node install-9209-patch.js
nvm install 16
OR nvm install 14.18.3
OR nvm use 16
NO AMPLIFY INSTALL
OR npm install -g @aws-amplify/cli@latest
OR yarn global add @aws-amplify/cli
OR npm config set user 0 && npm config set unsafe-perm true && npm install -g @aws-amplify/cli@latest
amplify --version
If you want a minor or patch update to v14, use npm install -g
.
node install-9209-patch.js
+ nvm install 14.18.3
+ npm install -g @aws-amplify/cli@latest
NO PATCH
+ nvm install 14.18.3
+ npm install -g @aws-amplify/cli@latest
Success π node v14.18.3 (npm v6.14.15)
version: 1
backend:
phases:
preBuild:
commands:
- "# Check Versions"
- node --version && npm --version && yarn --version && amplify --version
- "# Upgrade from 14.18.1 to 14.x"
- nvm install 14
- npm install -g @aws-amplify/cli@latest
- amplify --version
If you want to upgrade to v14 or v16 with yarn global add
, use the patch from @josefaidt above (https://github.com/josefaidt/9209/blob/main/scripts/patch.js).
node install-9209-patch.js
+ nvm install 14
+ yarn global add @aws-amplify/cli
node install-9209-patch.js
+ nvm install 16
+ yarn global add @aws-amplify/cli
Success π node v16.13.2 (npm v8.1.2) yarn@1.22.0
version: 1
backend:
phases:
preBuild:
commands:
- "# Check Versions"
- node --version && npm --version && yarn --version && amplify --version
- "# Upgrade from 14.18.1 to 16.x"
- node install-9209-patch.js
- nvm install 16
- yarn global add @aws-amplify/cli
- amplify --version
It is also possible to nvm _use_ 16
to access v16.13.0 since it is preinstalled but it is not recommended.
Using npm config set user 0 && npm config set unsafe-perm true && npm install -g @aws-amplify/cli@latest
to update amplify
Hey @chrismilleruk thank you for taking the time to include those details! The "packaged-node-modules" line added to the plugins JSON file should be all we need to resolve the "missing plugins..." error regardless of Node version, though I'm unsure what issues you may have encountered with the preinstalled Node version 16. In any event I'm glad to hear that you're unblocked π
Hey @josefaidt, No problem, sorry for the big blogpost..
Just checking again with a clear head and I can see I was wrong (copy paste error).
The patch in fact DID work for both v16 and v14 when using Yarn to upgrade amplify.
Thanks again Chris
I've updated the summary comment above for future readers.
Future readers:
I've managed to get next 14 working on amplify with cloudformation without needing to create custom image hosted on ECR...
Just use the public.ecr.aws/sam/build-nodejs[version].x:latest
images.
E.g., 18.x latest
EnvironmentVariables:
- Name: _CUSTOM_IMAGE
Value: "public.ecr.aws/sam/build-nodejs18.x:latest"
*Thanks to https://www.cloudypandas.ch/posts/for-when-you-need-a-custom-amplify-build-image/
I hope this finds you well :)
Before opening, please confirm:
How did you install the Amplify CLI?
npm & yarn
If applicable, what version of Node.js are you using?
node 16
Amplify CLI Version
7.3.3 - 7.3.5
What operating system are you using?
Amazon Linux
Amplify Categories
Not applicable
Amplify Commands
push, Not applicable
Describe the bug
In order to support typescript compilation in lambda functions (https://docs.amplify.aws/cli/function/build-options/) I need to upgrade
node
&npm
prior toamplifyPush --simple
(which has a dependency on@aws-amplify/cli
).Most folks seem to do
node
upgrades in thefrontEnd-preBuild
vianvm
and while this destroysamplify
cli, this goes unnoticed since thebackEnd-build
has already taken place.I tried a few different methods to get this working (below) but the simplest repro steps below (
nvm install 16
,yarn global add @aws-amplify/cli
create the following error:Other techniques attempted
nvm install 16
nvm install 16 --reinstall-packages-from=node
npm install -g @aws-amplify/cli@latest
yarn global add @aws-amplify/cli
@aws-amplify/cli@latest
in_LIVE_UPDATES
nvm install 16
via_LIVE_UPDATES
Side note. I had managed to workaround this issue previously by upgrading both
npm
and@aws-amplify/cli
in_LIVE_UPDATES
, which was just enough at the time, but that stopped working last week. I suspect the root cause of failure was a change (Tue 16th between 12:00 and 18:00 GMT) to run the npm live updates in parallel instead of in the supplied order. it's obviously not possible to upgrade npm while npm is installing something else so my hack failed. Regardless, the Cloning Repository step was marked as successful even though it failed with errors.Expected behavior
It should be possible to upgrade
node
usingnvm
and install@aws-amplify/cli@latest
prior toamplifyPush
Reproduction steps
GraphQL schema(s)
Log output
Additional information
Above is extracted from this build: eu-west-1#/d16xma096tmdk6/feature-sc-1422/21 The previous 20 builds may also be relevant as they were other attempts to solve the same issue.
Side Issue: Cloning Repository Step marked as successful
Logs from: eu-west-1#/d16xma096tmdk6/prod/15
``` # Put your logs below this line 2021-11-16T19:34:40.245Z [INFO]: Git SSH Key acquired 2021-11-16T19:34:40.321Z [INFO]: # Cloning repository: git@github.com:Akeero/akeero-admin.git 2021-11-16T19:34:40.374Z [INFO]: Agent pid 136 2021-11-16T19:34:40.396Z [INFO]: Identity added: /root/.ssh/git_rsa (/root/.ssh/git_rsa) 2021-11-16T19:34:40.583Z [INFO]: Cloning into 'akeero-admin'... 2021-11-16T19:34:41.097Z [INFO]: Warning: Permanently added the ECDSA host key for IP address '140.82.121.4' to the list of known hosts. 2021-11-16T19:34:43.412Z [INFO]: # Switching to commit: a723c66142bb1f4aca3306c0f418274016c3ca3f 2021-11-16T19:34:43.442Z [INFO]: Agent pid 149 2021-11-16T19:34:43.516Z [INFO]: Identity added: /root/.ssh/git_rsa (/root/.ssh/git_rsa) Note: switching to 'a723c66142bb1f4aca3306c0f418274016c3ca3f'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c