search

middyjs / middy

🛵 The stylish Node.js middleware engine for AWS Lambda 🛵
https://middy.js.org
MIT License
3.73k stars 376 forks source link

`dbManager` breaks Webpack bundling #440

Closed robdasilva closed 4 years ago

robdasilva commented 4 years ago

With the addition of dbManager middleware in version 0.32.0, I am no longer able to bundle my functions, as mssql is missing as a dependency of knex.

Even though I am not even using dbManager, as it's imported in the middlewares index file, I am affected, as soon as I use any middleware.

I switched back to version 0.30.5 for now, which is working fine.

lmammino commented 4 years ago

Hello @rschweizer! Thanks a lot for reporting this.

What is the exact error that you get? Which module bundler do you use? I thought mssql to be a peer dependency 😣

robdasilva commented 4 years ago

Hi @lmammino, Thank you for your quick reply!

The exact error I get is Error: Webpack compilation error with the stack output as listed below. But I actually need to correct myself: It's not just mssql which can't be resolved, but obviously all the DB client libraries, so oracledb, pg, etc. as well.

I've stripped some extraneous lines from the output below, namely the lines from the resolve, as those packages simply aren't installed. So that information is probably of no value. Should you need the entire output, just let me know ;)

Also, please let me know, if I can be of any other help!

./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'mssql' in './node_modules/knex/lib/dialects/mssql'
resolve 'mssql' in './node_modules/knex/lib/dialects/mssql'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/mssql)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/mssql/index.js 75:25-41
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'mssql/lib/base' in './node_modules/knex/lib/dialects/mssql'
resolve 'mssql/lib/base' in './node_modules/knex/lib/dialects/mssql'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/mssql)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/mssql/index.js 77:17-42
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'mssql/package.json' in './node_modules/knex/lib/dialects/mssql'
resolve 'mssql/package.json' in './node_modules/knex/lib/dialects/mssql'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/mssql)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/mssql/index.js 90:25-54
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/mysql/index.js
Module not found: Error: Can't resolve 'mysql' in './node_modules/knex/lib/dialects/mysql'
resolve 'mysql' in './node_modules/knex/lib/dialects/mysql'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/mysql)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/mysql/index.js 45:11-27
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/mysql2/index.js
Module not found: Error: Can't resolve 'mysql2' in './node_modules/knex/lib/dialects/mysql2'
resolve 'mysql2' in './node_modules/knex/lib/dialects/mysql2'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/mysql2)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/mysql2/index.js 37:11-28
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/oracle/index.js
Module not found: Error: Can't resolve 'oracle' in './node_modules/knex/lib/dialects/oracle'
resolve 'oracle' in './node_modules/knex/lib/dialects/oracle'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/oracle)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/oracle/index.js 51:11-28
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/oracledb/index.js
Module not found: Error: Can't resolve 'oracledb' in './node_modules/knex/lib/dialects/oracledb'
resolve 'oracledb' in './node_modules/knex/lib/dialects/oracledb'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/oracledb)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/oracledb/index.js 42:19-38 382:19-38
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/postgres/index.js
Module not found: Error: Can't resolve 'pg' in './node_modules/knex/lib/dialects/postgres'
resolve 'pg' in './node_modules/knex/lib/dialects/postgres'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/postgres)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/postgres/index.js 64:11-24
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/redshift/index.js
Module not found: Error: Can't resolve 'pg' in './node_modules/knex/lib/dialects/redshift'
resolve 'pg' in './node_modules/knex/lib/dialects/redshift'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/redshift)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/redshift/index.js 64:11-24
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/postgres/index.js
Module not found: Error: Can't resolve 'pg-query-stream' in './node_modules/knex/lib/dialects/postgres'
resolve 'pg-query-stream' in './node_modules/knex/lib/dialects/postgres'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/postgres)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/postgres/index.js 219:56-82
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/sqlite3/index.js
Module not found: Error: Can't resolve 'sqlite3' in './node_modules/knex/lib/dialects/sqlite3'
resolve 'sqlite3' in './node_modules/knex/lib/dialects/sqlite3'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/sqlite3)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/sqlite3/index.js 46:11-29
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js

./node_modules/knex/lib/dialects/mssql/index.js
Module not found: Error: Can't resolve 'tedious' in './node_modules/knex/lib/dialects/mssql'
resolve 'tedious' in './node_modules/knex/lib/dialects/mssql'
  Parsed request is a module
  using description file: ./node_modules/knex/package.json (relative path: ./lib/dialects/mssql)
    resolve as module
      ...
 @ ./node_modules/knex/lib/dialects/mssql/index.js 73:16-34
 @ ./node_modules/knex/lib/dialects sync ^\.\/.*\/index\.js$
 @ ./node_modules/knex/lib/knex.js
 @ ./node_modules/knex/lib/index.js
 @ ./node_modules/knex/knex.js
 @ ./node_modules/middy/src/middlewares/dbManager.js
 @ ./node_modules/middy/src/middlewares/index.js
 @ ./node_modules/middy/middlewares.js
lmammino commented 4 years ago

Feels more of a webpack issue. I would bet that knex code has a bunch of

if (something) {
  require('some-db-driver')
}

webpack ignores the if statements and tries to bundle all the required files that can find in the source code.

Here there might be a solution specific to webpack: https://github.com/knex/knex/issues/1446#issuecomment-537715431

lmammino commented 4 years ago

Let me know if this helps!

robdasilva commented 4 years ago

Thanks, @lmammino! Sure, that works. Much like defining those packages as externals or similar solutions.

Since I am not using dbManager at all – or any database connection for that matter – it feels kind of weird though. But I think I'll go for the mentioned approach for now.

Thanks, again! Cheers 🍻

lmammino commented 4 years ago

@rschweizer let's thank @gsingh1, @willfarrell, @theburningmonk and everyone else that took care to make this happen quickly :)

hsz commented 4 years ago

Works great, thanks!

robdasilva commented 4 years ago

Absolutely, @lmammino.

Thanks to @gsingh1, @willfarrell, @theburningmonk for the quick fix and everyone else, who might have helped shaping it :+1: