brianc / node-pg-cursor

Query cursor extension for node-postgres
78 stars 30 forks source link

Portal labelling change breaks knex migrations #56

Closed issa-tseng closed 4 years ago

issa-tseng commented 4 years ago

so, i have some migrations that opens a stream to generate some work, and then goes on later to run some alter statements. something like this:

  const work = [];
  for await (const form of db.select('*').from('forms').stream()) {
    work.push(
      db.insert({
        formId: form.id,
        xml: form.xml,
        hash: form.hash, sha: shasum(form.xml), sha256: sha256sum(form.xml),
        version: form.version,
        createdAt: form.createdAt
      }).into('form_defs')
    );
  }
  await Promise.all(work);

  /* a lot of other stuffs that all run fine.. */

  await db.schema.table('forms', (forms) => {
    forms.integer('currentDefId');

    forms.foreign('currentDefId').references('form_defs.id');

    forms.dropUnique([ 'xmlFormId', 'version', 'projectId' ]);
    forms.dropColumn('xml');
    forms.dropColumn('version');
    forms.dropColumn('hash');
  });

this all works fine with pg-query-stream 1.1.2 / pg-cursor 1.3.0. but if i upgrade both to 2.0.0, i get the following error:

  1) "before all" hook: initialize:
     alter table "forms" add column "currentDefId" integer - cannot ALTER TABLE "forms" because it is being used by active queries in this session
  error: cannot ALTER TABLE "forms" because it is being used by active queries in this session
      at Connection.parseE (node_modules/pg/lib/connection.js:604:11)
      at Connection.parseMessage (node_modules/pg/lib/connection.js:401:19)
      at Socket.<anonymous> (node_modules/pg/lib/connection.js:121:22)
      at addChunk (_stream_readable.js:294:12)
      at readableAddChunk (_stream_readable.js:275:11)
      at Socket.Readable.push (_stream_readable.js:210:10)
      at TCP.onStreamRead (internal/stream_base_commons.js:166:17)

if i manually dig into node_modules and set a portal of '' as it was before, everything starts working again. i have postgres dumping all queries to a logfile and there is no discernible difference in the queries being executed.

is there some way i can work around this problem? i am only trying to upgrade because there is a memory leak issue i've been having with Buffers returned over the stream that seems to go away magically when i upgrade to 2.0.0. but when i do, the migrations don't work anymore.

thanks! i am happy to open a PR given guidance on where to go with this. i am not sure why setting the portal label causes a problem.