SNOW-765513: optional peer dependencies for different clouds

snowflakedb / snowflake-connector-nodejs

NodeJS driver

Apache License 2.0

121 stars 125 forks source link

SNOW-765513: optional peer dependencies for different clouds #449

Open brianzinn opened 1 year ago

brianzinn commented 1 year ago

What did you do? I'm running snowflake-sdk in google cloud function.
What did you expect to see? No messages for AWS
What did you see instead? (node:1) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. Please migrate your code to use AWS SDK for JavaScript (v3)

Is there any consideration to make AWS/Azure as optional peer dependencies? This would:

reduce bundle size
allow the folks requesting AWS v3 to choose their version #365

sfc-gh-dszmolka commented 1 year ago

thank you for submitting this feature request ! we'll see how it would be feasible to implement. thank you for bearing with us !

cleve-fauna commented 1 year ago

Just providing more data here :)

For my use case - I would like to use the snowflake-sdk parts that communicate with snowflake - and don't need to use any of the bundled in cloud SDKs.

kirillgroshkov commented 1 year ago

For my use case - I would like to use the snowflake-sdk parts that communicate with snowflake - and don't need to use any of the bundled in cloud SDKs.

Same. That's a perfect case for peerDependencies.

kirillgroshkov commented 1 year ago

If this is not fixed - I guess as a community we'll need to settle on a fork that would have these extra deps as peerDeps. We're getting close to forking it BTW.

kirillgroshkov commented 8 months ago

Heads up, since this issue is not fixed - I've started working on a fork for our organization, to get rid of unused dependencies that bloat our lockfiles and node_modules (and CI with npm warnings).

I've now pushed the first version of the fork and it has passed our org internal tests - all our use cases of snowflake-sdk still work (as we don't require AWS sdk, Azure, etc).

Not encouraging anyone to use it (better to fix the original package), but here's our published example: https://www.npmjs.com/package/@naturalcycles/snowflake-sdk

yarn.lock diff looks like this:

brianzinn commented 8 months ago

thanks @kirillgroshkov - the Node.js support from snowflake appears the least supported (https://developers.snowflake.com/drivers-and-libraries/). I wrote also years ago a Node.js snowpipe SDK (https://github.com/brianzinn/snowflake-ingest-node). I haven't been following the updates lately, but it looks like they are encouraging streams now and I don't see any snowpipe support for any language anymore - looks like they are using Spark/snowpark instead of an API, so will be even less Node.js compatibility!? Probably makes sense from a data team/processing perspective, but not necessarily as part of a company ecosystem in my case even though I am proficient in Python and Java. Then again... I am not following their developer strategy!!

kirillgroshkov commented 8 months ago

Today I stepped one level deeper and found more unused/unnecessary dependencies to remove from package.json (or move to devDependencies):

larryboymi commented 2 months ago

Just providing more data here :)

For my use case - I would like to use the snowflake-sdk parts that communicate with snowflake - and don't need to use any of the bundled in cloud SDKs.

Ditto