Open saimonmoore opened 4 years ago
This seems to be an upstream issue β I can replicate this issue when I try to install the lz4
package.
$ yarn add lz4
yarn add v1.22.4
[1/4] π Resolving packages...
[2/4] π Fetching packages...
error https://registry.yarnpkg.com/lz4/-/lz4-0.6.3.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, link '/Users/gp/Library/Caches/Yarn/v6/npm-lz4-0.6.3-78df6bb69a36d7db6c2e849494876ba6e38e66d6-integrity/node_modules/lz4/build/Release/obj.target/build/Release/lz4.node' -> '/Users/gp/Library/Caches/Yarn/v6/npm-lz4-0.6.3-78df6bb69a36d7db6c2e849494876ba6e38e66d6-integrity/node_modules/lz4/build/Release/obj.target/lz4.node'"
@saimonmoore Please open an issue in the lz4
project's issue tracker.
Was already opened: https://github.com/pierrec/node-lz4/issues/97
It's due to a breaking change in yarn v1.22.
Running yarn policies set-version 1.21
fixed the problem for me.
@saimonmoore Can you confirm the the workaround posted above by @leoparis89 fixes this issue?
yarn policies set-version 1.21
Can you folks try the latest v2 beta release? See https://github.com/indix/kafkajs-lz4/pull/7#issuecomment-738652072
yarn add kafkajs-lz4@2.0.0-beta.0
Hey paambati
We are getting a typeerror, we tried both version 1.2.1 and the latest beta - 2.0.0-beta:
TypeError: kafkajs_lz4_1.default is not a constructor
Thanks
@link2malkeet Can you share the version of Node and the relevant piece of code where youβre using this package?
Thanks for coming so quickly,
That's resolved, actually we followed the documentation and found the issue with the way we need to call codec on the instance of LZ4Codec
with this code snippet, we manage to create an instance of LZ4Codec
const LZ4Codec = require('kafkajs-lz4'); // import LZ4Codec from
kafkajs-lz4did not work. const LZ4Instance = new LZ4Codec(); CompressionCodecs[CompressionTypes.LZ4] = LZ4Instance.codec;
But
After this configuration, still we are getting the same old error saying
LZ4 compression not implemented
.
Usage looks very simple but could not get it working yet. π Thanks Mel
@link2malkeet This should work though, I'm not sure of the root-cause of your issue.
Can you follow the examples (or even our tests); they're fairly straightforward and are known to work.
If that doesn't work, I'm afraid we'll need to know more about your setup and your code.
Well our setup is pretty straight forward. Before we create Producer and Consumer instances, we simply add the compression configuration as suggested so build and deployment works fine but when consumer listens to the kafka topic(which has messages compressed with LZ4), the subscription throws this error saying LZ4 compression not implemented
Node: v14.15.0
Yarn: v1.21.1
can you point out where we are supposed to add LZ4 configuration?
where we are supposed to add LZ4 configuration?
@link2malkeet They're supposed to be added to the producer.send
method. You can check our tests (see https://github.com/indix/kafkajs-lz4/blob/5efecce6fc8e746d43bc21298e169eb62eb65b5a/test/index.test.ts#L59-L112) and the official documentation (see https://kafka.js.org/docs/producing).
Finally it worked.
Instead of using this library, we simple took the index.ts
file source.
Seems this is culprit
there are 2 things exporting LZ4Codec
export default class LZ4Codec
vs module.exports = LZ4Codec;
Can you guys please update it with the change so that we could use this libraray with the required fix?
@link2malkeet I'm sorry, but I don't think that's the right approach. I'd recommend you read up about CommonJS and ES6 imports and how they're different.
For a proper solution to your issue, like I've said before, I'd need more information β that includes your Node version, TS version (if you're using any), Babel config (if you're using any) and more relevant code (how you're importing this package, how/where you're using the produce & consume methods, etc.)
If you'd like to pursue this, I'd recommend you open a new issue as yours seems to be different from this.
Same issues when i use nestjs monorepo.
I fininally solve it by use const LZ4Codec = require('kafkajs-lz4')
, and here is my tsconfig.json
:
{
"compilerOptions": {
"module": "CommonJS",
"target": "ES2017"
"Other configs": "..."
}
}
Unable to install this package: