srdone
commented
3 years ago I'm seeing this issue - we've had to specify 2.17.0 in order for our deploys to work.
Closed lewgordon closed 3 years ago
srdone
commented
3 years ago I'm seeing this issue - we've had to specify 2.17.0 in order for our deploys to work.
srdone
commented
3 years ago I should add - we don't have a direct dependency on ajv, but we're using other packages that likely have dependencies on ajv.
medikoo
commented
3 years ago @lewgordon Thanks for report, Still this looks to me as issue in your setup, at least provided information shows that:
You use globally installed serverless v2.18.0 , which in context of /tmp/test fallbacks to it's locally installed serverless v2.11.1.
And it appears that local serverless installation in tmp/test/node_modules v2.11.1 is broken:
Note that v2.11.1 depends on ajv-keywords@^3.5.2 -> https://github.com/serverless/serverless/blob/bde334c10f33a813a16e158832eb61c56e77d822/package.json#L30
and requires it in lib/class/ConfigSchemaHandler/index.js:
https://github.com/serverless/serverless/blob/bde334c10f33a813a16e158832eb61c56e77d822/lib/classes/ConfigSchemaHandler/index.js#L102
Still what it receives in your case is ajv-keywords v4, we can tell that by resolved require path ajv-keywords/dist/index.js exposed in your stack trace which is specific to v4 -> https://npmview.now.sh/ajv-keywords@4 while v3 looks as https://npmview.now.sh/ajv-keywords@3.
How did you install serverless locally? Is this referenced in package.json of /tmp/test?, because if it's not and then e.g. ajv-keywords@4 was installed in context of /tmp/test it possibly could break the unlisted serverless installation
srdone
commented
3 years ago In our case, serverless is installed as a dev dependency of our project. We are not using ajv directly (though one or more of our dependencies are bringing it in on install).
claylevering
commented
3 years ago As a note, had the same issues starting in 2.18.0 - Our team uses the serverless library in our dev dependencies and of course we have some library dependencies on AJV but at least in the app we're working with now, the list of dependencies is fairly minimal:
...
"devDependencies": {
"jest": "^26.6.3",
"serverless": "^2.17.0",
"serverless-offline": "^6.8.0",
"serverless-plugin-git-variables": "^5.1.0"
},
"dependencies": {
"INTERNAL LIBRARY": "^1.0.0-alpha.6"
}2.17.0 works, 2.18.0 fails with the AJV errors listed above.
claylevering
commented
3 years ago I should note we never leverage a global serverless installation. We rely on nvm and per-repo version locking with either npx sls or npm scripts to call any serverless executions (e.g. npm run deploy:production ==> serverless deploy --stage production)
medikoo
commented
3 years ago It'll be great to find out how this broken state happens.
At least in @lewgordon it looks as broken installation issue, which is not an issue in serverless library itself (as it doesn't install by itself, but via tools as npm or yarn)
Even if service on its own depends on ajv or ajv-keywords (or has dependencies that depend on it) at different versions than ones on which serverless depends. Then installer should ensure versions which serverless expects in its servelress/node_modules folder (that's how npm and yarn based installation works)
FergusMcGlynn
commented
3 years ago I am running into this issue as well. I actually have two projects that use serverless as a dev dependency - I upgraded both to use version 2.18.0 yesterday in order to fix the axios security vulnerability - for one project there are no problems running serverless deploy, but for the other I get the same error as the original poster: Cannot find module 'ajv/dist/compile/codegen'.
Running npm ls ajv on both projects, I see that on the project where serverless deploy fails, I have an UNMET PEER DEPENDENCY:
$ npm ls ajv
+-- jest@26.6.3
| `-- @jest/core@26.6.3
| `-- jest-config@26.6.3
| `-- jest-environment-jsdom@26.6.2
| `-- jsdom@16.4.0
| `-- request@2.88.2
| `-- har-validator@5.1.5
| `-- ajv@6.12.6 deduped
+-- serverless@2.18.0
| +-- UNMET PEER DEPENDENCY ajv@7.0.3
| `-- ajv-formats@1.5.1
| `-- ajv@7.0.3
`-- xo@0.33.1
`-- eslint@7.17.0
+-- @eslint/eslintrc@0.2.2
| `-- ajv@6.12.6 deduped
+-- ajv@6.12.6
`-- table@6.0.7
`-- ajv@7.0.3But on the other project, where serverless deploy works fine, no peer dependency issues, even though the conditions regarding ajv look very similar:
$ npm ls ajv
+-- @percy/script@1.1.0
| `-- @percy/agent@0.28.6
| `-- percy-client@3.8.0
| `-- request@2.88.2
| `-- har-validator@5.1.5
| `-- ajv@6.12.6 deduped
+-- @university-of-york/esg-lib-pattern-library-react-components@6.1.4
| `-- babel-plugin-react-css-modules@5.2.6
| `-- ajv@6.12.6
+-- next@10.0.5
| +-- sass-loader@10.0.5
| | `-- schema-utils@3.0.0
| | `-- ajv@6.12.6 deduped
| +-- schema-utils@2.7.1
| | `-- ajv@6.12.6 deduped
| `-- webpack@4.44.1
| +-- ajv@6.12.6 deduped
| +-- schema-utils@1.0.0
| | `-- ajv@6.12.6 deduped
| `-- terser-webpack-plugin@1.4.5
| `-- schema-utils@1.0.0
| `-- ajv@6.12.6 deduped
+-- serverless@2.18.0
| +-- ajv@7.0.3
| `-- ajv-formats@1.5.1
| `-- ajv@7.0.3
`-- xo@0.33.1
`-- eslint@7.17.0
+-- @eslint/eslintrc@0.2.2
| `-- ajv@6.12.6 deduped
+-- ajv@6.12.6 deduped
`-- table@6.0.7
`-- ajv@7.0.3 I don't understand why there would be an unmet peer dependency in one project and not the other. Neither explicitly includes ajv as a dependency. Also, it looks to me like serverless has ajv as an explicit dependency, so I'm confused by the peer dependency issue here.
lewgordon
commented
3 years ago Sorry about that @medikoo ! I agree there was something wrong in my setup. I reproduced the issue I was seeing again, though. I believe there might have been a typo in my original post. Here are the files for the most recent reproduction. Let me know if you need additional information or find anything I've missed.
This time I already ran it in the context of a container to isolate the filesystem. I used node with tag 12.
serverless.ymlsls package outputInstalled version
Framework Core: 2.18.0 (local)
Plugin: 4.4.2
SDK: 2.3.2
Components: 3.4.6history outputlewgordon
commented
3 years ago Here's another example of probably a more realistic scenario, where there is a dependency on ajv@^6. Again using a node:12 container.
package.jsonserverless.ymlsls package outputhistory outputmedikoo
commented
3 years ago @lewgordon From what I see it's an installation error. How do you install the project ? Is it simply npm install ? If so, what's the version of npm?
In your output we can see that ajv-keyword@4 which has a peer dependency set to ajv@^7.0.0 is installed top-level aside of ajv at v6.
It looks as a clear bug of installer, as with your configuration, ajv-keyword should be installed in serverless/node_modules aside of its ajv@7
lewgordon
commented
3 years ago Hey @medikoo, it's just a npm install. npm version is 6.14.10.
medikoo
commented
3 years ago @lewgordon I've just tested it locally, and indeed it happens with npm v6, but not with npm v7 which installs project as expected
It's a clear bug in npm v6 (it's probably unlikely they'll fix it in that branch)
FergusMcGlynn
commented
3 years ago npm v7 auto installs peer dependencies, whereas npm v6 does not. It might be this feature that makes it work under npm v7, rather than a bug fix for some behaviour in npm v6?
medikoo
commented
3 years ago Issue is not that peer dependency is not installed, but it's because dependency of serverless is installed in wrong location against its peer dependency which is also marked as dependency of serverless
medikoo
commented
3 years ago But it might be that with npm v6 support for peer dependencies was just informative, and in light of that, it's no wonder it cannot handle such setup properly.
@lewgordon workaround on your side could be to define ajv-keywords@3 dependency aside ajv@6. It'll ensure that ajv-keywords@4 will get installed in serverless/node_modules
srdone
commented
3 years ago Seeing as npm v6 is the current LTS version, and npm v7 is the development release, I believe serverless should not break under npm v6 just because the project where it is used happens to have ajv 6 as a dependency. Especially considering that the previous minor release does not break in this scenario.
That's assuming I understand the problem correctly.
Is there not a way the change to serverless that broke this scenario could be modified so that it does not break in these circumstances?
medikoo
commented
3 years ago @srdone this problem is not specific to latest changes.
Same would happen if you'd try to install ajv at v5 (or at v7) with previous version of serverless
The only "complete fix" could be to not use in serverless packages that declare peer dependencies at all, and imo that's not really viable.
I believe right approach is to acknowledge the specificity of a setup and find a way to install it properly (either use package manager that handles peer dependencies, or workaround it as I proposed above)
jakoritarleite
commented
3 years ago I was facing the same issue with, I temporarily solved it by installing ajv@7.0.3 and now it's added as dependency on my package.json:
...
"devDependencies": {
"serverless-step-functions": "^2.27.1"
},
"dependencies": {
"ajv": "^7.0.3",
"serverless": "^2.18.0",
"serverless-pseudo-parameters": "^2.5.0",
"serverless-python-requirements": "^5.1.0"
}It's just a "workaround" for those who need to deploy their applications, and problably will be fixed asap. By the way I'm running the serverless v2.18.0, npm v6.14.10 and node v10.19.0.
srdone
commented
3 years ago @medikoo - we are currently getting around this issue by pinning our dev dependency on serverless at v.2.17.0, so the problem does not exist there.
Note that we do not have any direct dependency on ajv - but rather another package we are using has it as a dependency.
FergusMcGlynn
commented
3 years ago I think there is something funny about version 2.18.0 of serverless. 3 separate users have posted here that they have run into the error Cannot find module 'ajv/dist/compile/codegen' since upgrading to 2.18.0, and they didn't have any errors with 2.17.0. Also, in the example I described in an earlier post, upgrading to 2.18.0 caused the error in one project but not in the other. If this were a straight up bug in npm v6, wouldn't both of them fail?
My projects don't have a direct dependency on ajv either.
medikoo
commented
3 years ago My projects don't have a direct dependency on ajv either.
I believe some other dependency in your setup relies on ajv at other version than serverless.
Note that this issue will happen if two dependencies rely on different version of ajv (and aside in any of the dependencies there's a dependency that marks ajv as peer).
We may downgrade back ajv to v6, but if your other dependency upgrades ajv internally to v7, again you will have same issue
FergusMcGlynn
commented
3 years ago In the project where upgrading to version 2.18.0 worked fine, there were two dependencies relying on different versions of ajv:
$ npm ls ajv
+-- @percy/script@1.1.0
| `-- @percy/agent@0.28.6
| `-- percy-client@3.8.0
| `-- request@2.88.2
| `-- har-validator@5.1.5
| `-- ajv@6.12.6 deduped
+-- @university-of-york/esg-lib-pattern-library-react-components@6.1.4
| `-- babel-plugin-react-css-modules@5.2.6
| `-- ajv@6.12.6
+-- next@10.0.5
| +-- sass-loader@10.0.5
| | `-- schema-utils@3.0.0
| | `-- ajv@6.12.6 deduped
| +-- schema-utils@2.7.1
| | `-- ajv@6.12.6 deduped
| `-- webpack@4.44.1
| +-- ajv@6.12.6 deduped
| +-- schema-utils@1.0.0
| | `-- ajv@6.12.6 deduped
| `-- terser-webpack-plugin@1.4.5
| `-- schema-utils@1.0.0
| `-- ajv@6.12.6 deduped
+-- serverless@2.18.0
| +-- ajv@7.0.3
| `-- ajv-formats@1.5.1
| `-- ajv@7.0.3
`-- xo@0.33.1
`-- eslint@7.17.0
+-- @eslint/eslintrc@0.2.2
| `-- ajv@6.12.6 deduped
+-- ajv@6.12.6 deduped
`-- table@6.0.7
`-- ajv@7.0.3 As I say, this one works and does not run into the error Cannot find module 'ajv/dist/compile/codegen'.
srdone
commented
3 years ago In our project, before upgrading to 2.18.0, this is the result on running npm ls ajv:
├─┬ objection@2.2.5
│ └── ajv@6.12.6
├─┬ redoc-cli@0.9.7
│ └─┬ redoc@2.0.0-rc.24
│ └─┬ swagger2openapi@5.4.0
│ └─┬ oas-validator@3.4.0
│ └── ajv@5.5.2
├─┬ request@2.88.2
│ └─┬ har-validator@5.1.5
│ └── ajv@6.12.6 deduped
├─┬ serverless@2.17.0
│ └── ajv@6.12.6 deduped
└─┬ standard@14.3.4
└─┬ eslint@6.8.0
├── ajv@6.12.6 deduped
└─┬ table@5.4.6
└── ajv@6.12.6 dedupedAfter upgrading to 2.18.0, the result of npm ls ajv becomes:
├─┬ objection@2.2.5
│ └── ajv@6.12.6
├─┬ redoc-cli@0.9.7
│ └─┬ redoc@2.0.0-rc.24
│ └─┬ swagger2openapi@5.4.0
│ └─┬ oas-validator@3.4.0
│ └── ajv@5.5.2
├─┬ request@2.88.2
│ └─┬ har-validator@5.1.5
│ └── ajv@6.12.6 deduped
├─┬ serverless@2.18.0
│ ├── UNMET PEER DEPENDENCY ajv@7.0.3
│ └─┬ ajv-formats@1.5.1
│ └── ajv@7.0.3
└─┬ standard@14.3.4
└─┬ eslint@6.8.0
├── ajv@6.12.6 deduped
└─┬ table@5.4.6
└── ajv@6.12.6 dedupedThis suggests to me that serverless expects ajv@7.x to be installed in any project where it is used. That's how peer dependencies are expected to work in npm - if a package is declared as a peer dependency, npm does not automatically install it but instead warn the user of the package that it is missing a peer dependency. In this case, I get the warning:
npm WARN ajv-keywords@4.0.0 requires a peer of ajv@^7.0.0 but none is installed. You must install peer dependencies yourself.The user is then expected to install the missing peer dependency in their project where the package requiring the peer dependency is used.
I can, of course, install ajv@7.0.0 as a dev dependency alongside serverless to fix the issue I am experiencing (which is the standard solution in this scenario). However, it seems to me that since serverless is intended to run independently of the project and it requires another package to operate properly, it should not declare that dependency as a peer dependency - it should declare it as a dependency.
medikoo
commented
3 years ago it should not declare that dependency as a peer dependency - it should declare it as a dependency.
@srdone serverless declares ajv@7 as dependency:
If in your case installedserverless has no visiblity of ajv it means that installation is broken (and installer is to blame not serverless)
SamuelHaleMN
commented
3 years ago Just to chime in... this odd issue also threw our CI/CD build pipeline for a loop.
Clearly, whether this is "intended behavior" or not, this change to the ajv dependency has caused quite a few issues.
medikoo
commented
3 years ago Clearly, whether this is "intended behavior" or not, this change to the ajv dependency has caused quite a few issues.
I believe it caused issues to cases where (1) service depends directly on ajv and (2) service was installed with installer that has limited support for handling peer dependencies
Note that all we did is upgrading ajv, downgrading it back won't ensure that same issue cannot happen.
FergusMcGlynn
commented
3 years ago I think @medikoo is correct that this issue has been caused by serverless version 2.18.0 upgrading its dependencies. Specifically, I think it's the upgrade of ajv-keywords to version ^4.0.0 that's brought this problem to the surface - that's the package that has the peer dependency on ajv version ^7.0.3.
This looks to be an example of a bug in npm that's been around for a long time. That issue is almost 2 years old and was never resolved by the look of things. It might have been resolved by npm version 7, but I can't find confirmation of that anywhere. The person who raised that issue did a great job of reproducing the problem, and you can see that the circumstances in their demo project are very similar to the case that we're seeing here. Just replace webpack in their example with serverless in ours and it looks like the very same situation.
In which case, I suppose the options are to do one of the following:
ajv version ^7.0.3 as a dev dependency to your projectmedikoo
commented
3 years ago AJV will be reverted to v6 starting with next release of Framework, it's due to performance issues with observed which appear to not be easily fixable, for more info check:
This means that this problem will be fixed for you, unless you've upgraded your services to rely on AJV v7, as then you'll face the same problem with same installer when serverless depends on AJV v6
claylevering
commented
3 years ago This seems fixed as far as I can tell, I think we can close it?
Not sure if this is something that just can't be helped, but it seems that if a NodeJS project has a dependency on
ajv^6it'll cause the packaging step to fail. I believe this was caused by the bump toajv^7. I know that many projects still depend on the earlier version, so it might break a lot of projects.Steps to reproduce:
sls initajv^6vianpm i ajv@^6sls package
```yaml # Welcome to Serverless! # # This file is the main config file for your service. # It's very minimal at this point and uses default values. # You can always add more config options for more control. # We've included some commented out config examples here. # Just uncomment any of them to get that config option. # # For full config options, check the docs: # docs.serverless.com # # Happy Coding! service: test # app and org for use with dashboard.serverless.com #app: your-app-name #org: your-org-name # You can pin your service to only deploy with a specific Serverless version # Check out our docs for more details frameworkVersion: '2' provider: name: aws runtime: nodejs12.x # you can overwrite defaults here # stage: dev # region: us-east-1 # you can add statements to the Lambda function's IAM Role here # iamRoleStatements: # - Effect: "Allow" # Action: # - "s3:ListBucket" # Resource: { "Fn::Join" : ["", ["arn:aws:s3:::", { "Ref" : "ServerlessDeploymentBucket" } ] ] } # - Effect: "Allow" # Action: # - "s3:PutObject" # Resource: # Fn::Join: # - "" # - - "arn:aws:s3:::" # - "Ref" : "ServerlessDeploymentBucket" # - "/*" # you can define service wide environment variables here # environment: # variable1: value1 # you can add packaging information here #package: # include: # - include-me.js # - include-me-dir/** # exclude: # - exclude-me.js # - exclude-me-dir/** functions: hello: handler: handler.hello # The following are a few example events you can configure # NOTE: Please make sure to change your handler code to work with those events # Check the event documentation for details # events: # - http: # path: users/create # method: get # - websocket: $connect # - s3: ${env:BUCKET} # - schedule: rate(10 minutes) # - sns: greeter-topic # - stream: arn:aws:dynamodb:region:XXXXXX:table/foo/stream/1970-01-01T00:00:00.000 # - alexaSkill: amzn1.ask.skill.xx-xx-xx-xx # - alexaSmartHome: amzn1.ask.skill.xx-xx-xx-xx # - iot: # sql: "SELECT * FROM 'some_topic'" # - cloudwatchEvent: # event: # source: # - "aws.ec2" # detail-type: # - "EC2 Instance State-change Notification" # detail: # state: # - pending # - cloudwatchLog: '/aws/lambda/hello' # - cognitoUserPool: # pool: MyUserPool # trigger: PreSignUp # - alb: # listenerArn: arn:aws:elasticloadbalancing:us-east-1:XXXXXX:listener/app/my-load-balancer/50dc6c495c0c9188/ # priority: 1 # conditions: # host: example.com # path: /hello # Define function environment variables here # environment: # variable2: value2 # you can add CloudFormation resource templates here #resources: # Resources: # NewResource: # Type: AWS::S3::Bucket # Properties: # BucketName: my-new-bucket # Outputs: # NewOutput: # Description: "Description for the output" # Value: "Some output value" ```serverless.yml
``` Serverless: Running "serverless" installed locally (in service node_modules) Error -------------------------------------------------- Error: Cannot find module 'ajv/dist/compile/codegen' Require stack: - /tmp/test/node_modules/ajv-keywords/dist/definitions/typeof.js - /tmp/test/node_modules/ajv-keywords/dist/keywords/typeof.js - /tmp/test/node_modules/ajv-keywords/dist/keywords/index.js - /tmp/test/node_modules/ajv-keywords/dist/index.js - /tmp/test/node_modules/serverless/lib/classes/ConfigSchemaHandler/index.js - /tmp/test/node_modules/serverless/lib/Serverless.js - /usr/local/lib/node_modules/serverless/lib/Serverless.js - /usr/local/lib/node_modules/serverless/scripts/serverless.js - /usr/local/lib/node_modules/serverless/bin/serverless.js at Function.Module._resolveFilename (internal/modules/cjs/loader.js:957:15) at Function.Module._load (internal/modules/cjs/loader.js:840:27) at Module.require (internal/modules/cjs/loader.js:1019:19) at require (internal/modules/cjs/helpers.js:77:18) at Object.sls packageoutputInstalled version