Possible to reduce node package size?

[dmb0058]

Describe the bug AWS Lambda functions are restricted to 250Mb including layers. Packaging the aws-sdk, faiss-node, fs, langchain, openai and adm-zip blows the limit.

I've pulled out only the aws sub-packages I need (s3 and secrets-manager) and the fs and adm-zip packages are tiny, but faiss-node is pretty big. I see it has Readme files and stuff, so I guess it's not 'minimal' - any chance of a slimmed down npm package?

Environment:

• Operating system: AWS Lambda
• Nodejs Version: v18.x
• Package Version: latest

To Reproduce Steps to reproduce the behavior:

1. Zip the nodejs/node_modules hierarchy
2. Attempt to create a Lambda layer
3. Error is "Layer max. size is 250Mb" (paraphrased)

Expected behavior The layer is correctly created.

[ewfian]

@dmb0058 Now use MKL on linux and openblas on musllinux. We can consider replacing it with openblas for reducing size.

[dmb0058]

Excellent news! This will make a lot of difference, thanks.

I'll happily try this once the langchainjs issue is resolved.

On Wed, 12 Jul 2023, 08:27 Fan, @.***> wrote:

@dmb0058 https://github.com/dmb0058 Now use MKL on linux and openblas on musllinux. We can consider replacing it with openblas for reducing size.

[image: image] https://user-images.githubusercontent.com/12423122/252887542-7ba8f9a2-08fd-4d14-96d2-172ca33346fc.png

— Reply to this email directly, view it on GitHub https://github.com/ewfian/faiss-node/issues/26#issuecomment-1631990845, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXCQJXK2XUUY4N2SMHT73XPZGWTANCNFSM6AAAAAA2F7YBTY . You are receiving this because you were mentioned.Message ID: @.***>

[dmb0058]

Just checking if the latest version is the one on https://www.npmjs.com/package/faiss-node, or need to install from git?

[ewfian]

@dmb0058 I will publish a new version to npm soon.

You can check pre-release version here: https://github.com/ewfian/faiss-node/releases/tag/v0.2.3-alpha.0

[ewfian]

@dmb0058 The new version v0.2.3 was released.

[dmb0058]

That's great news, thanks for all your help :)

On Mon, 7 Aug 2023, 02:14 Fan, @.***> wrote:

@dmb0058 https://github.com/dmb0058 The new version v0.2.3 was released.

— Reply to this email directly, view it on GitHub https://github.com/ewfian/faiss-node/issues/26#issuecomment-1667044483, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXCQIZOJS6DHK2JXMGJP3XUA6QBANCNFSM6AAAAAA2F7YBTY . You are receiving this because you were mentioned.Message ID: @.***>