TypeError: Invalid query format when massive & pg-promise match demux-js-postgres module

[jasonbert]

I'm receiving the TypeError: Invalid query format. error when my init migration is run. The setup prior to that has created tables in Postgres, so it's definitely able to do some queries.

PS C:\Work\bb\git\enterprise.demux> node src/index.js                                                                                                                                                                                                    
-- Starting Demux --

[ { migrations: [ [Migration] ], sequenceName: 'init' } ]
{"name":"demux","hostname":"localhost","pid":23216,"level":50,"err":{"message":"The proper initialization has not occurred. Failed to migrate the postgres database.","name":"NotInitializedError","stack":"NotInitializedError: The proper initialization has not occurred. Failed to migrate the postgres database.
    at MassiveActionHandler.<anonymous> (C:\\Work\\bb\\git\\enterprise.demux\\node_modules\\demux-postgres\\dist\\MassiveActionHandler.js:97:23)
    TypeError: Invalid query format.
    at Database.$query (C:\\Work\\bb\\git\\enterprise.demux\\node_modules\\pg-promise\\lib\\query.js:104:21)
    at Database.<anonymous> (C:\\Work\\bb\\git\\enterprise.demux\\node_modules\\pg-promise\\lib\\query.js:259:23)
    at config.$npm.connect.pool.then.db (C:\\Work\\bb\\git\\enterprise.demux\\node_modules\\pg-promise\\lib\\database.js:326:42)
    at process._tickCallback (internal/process/next_tick.js:68:7)"},"msg":"The proper initialization has not occurred. Failed to migrate the postgres database.","time":"2019-08-22T15:05:10.796Z","v":0}
{"name":"demux","hostname":"localhost","pid":23216,"level":30,"msg":"Indexing unexpectedly paused due to an error.","time":"2019-08-22T15:05:10.797Z","v":0}

index.js

console.log("");
console.log("-- Starting Demux --")
console.log("");

const massive = require('massive')
const { BaseActionWatcher } = require("demux")
const { MassiveActionHandler } = require("demux-postgres")
const { NodeosActionReader } = require("demux-eos")

const handlerVersions = require("./handlerVersions")
const migrationSequences = require("./migrationSequences")

console.log(migrationSequences);

const dbConfig = {
    host: '127.0.0.1',
    port: 5432,
    database: 'enterprise',
    user: 'postgres',
    password: 'docker'
}

massive(dbConfig).then((db) => {
    const actionReader = new NodeosActionReader("http://localhost:8888", 0)
    const actionHandler = new MassiveActionHandler(
        handlerVersions,
        db,
        "public",
        migrationSequences
    )
    const actionWatcher = new BaseActionWatcher(actionReader, actionHandler, 500)
    actionWatcher.watch()
})

migrationSequences/index.js

const { Migration } = require('demux-postgres')

const createTables = new Migration(
    "createTables", // name
    "public", // schema
    "migrationSequences/createTables.sql", // SQL file
)

module.exports = [{
    migrations: [createTables],
    sequenceName: "init"
}]

migrationSequences/createTables.sql

CREATE TABLE IF NOT EXISTS "${schema^}"."match_type" (
    "id" CHAR(64) PRIMARY KEY,
    "type" VARCHAR NOT NULL,
    "max_oppoonents" INTEGER NOT NULL
);

[vitaly-t]

The error means: pg-promise received a request from Massive.js that's not a real query, possibly a null, undefined or an empty string. In other words, at some point Massive.js failed to prepare a valid query, but still tried to execute it via pg-promise, which in turn threw the error.

[jasonbert]

Thanks for shedding some light on the potential issue, however I'm not sure where a query like that would be coming from?

[vitaly-t]

There it says in the error:

node_modules\demux-postgres\dist\MassiveActionHandler.js:97:23

It is doing there something that makes Massive.js execute an empty or missing query.

[jasonbert]

Still digging into this issue, but it appears to be because of the Cyan Audit SQL file that needs to be run. In the generic pg-promise $query function there's a check to see if the query is a string, but in this case the variable is an object.

Looks like PG Promise isn't able to work out what to do with that query file and therefore doesn't change the query object in the actual query string.

[jasonbert]

Looks like this is a duplicate of issue https://github.com/EOSIO/demux-js-postgres/issues/67

[vitaly-t]

Yes, but from what I said above, you are looking at too deep. The issue is above pg-promise where such an invalid object gets passed in through Massive.js somehow, an object that does not represent a valid query. pg-promise correctly throwing on it is just the final consequence of it, as the higher layers fail to handle the situation.

[jasonbert]

Just to be clear, I haven't said pg-promise is the problem ;). And looking this deep led me to the same issue someone else faced.

[jasonbert]

Added comments to closed issue #67, but reopening this one as it's closed. I had to remove .default in the generated index.js file for the module. There's no default property when you do require('massive') as that exports the default and is then in massive_1. Obviously this isn't a fix and needs to be corrected in the TypeScript file https://github.com/EOSIO/demux-js-postgres/blob/develop/src/index.ts

[jasonbert]

Someone has kindly made a PR to fix this issue :)

https://github.com/EOSIO/demux-js-postgres/pull/83