timgit / pg-boss

Queueing jobs in Postgres from Node.js like a boss
MIT License
2.04k stars 157 forks source link

PgBoss never exits #233

Closed romandecker closed 3 years ago

romandecker commented 3 years ago

Hey there!

thanks for this great library! I've found an issue that pg-boss never seems to free the pg-pool when scheduling is enabled. The following snippet never exits:

const PgBoss = require('pg-boss');

(async function main() {
    boss = new PgBoss({
        host: 'localhost',
        port: 5432,
        database: 'postgres',
        user: 'postgres',
        retentionMinutes: 30,
        max: 1,
        // noScheduling: true  // <- uncomment this to make it work
    });

    console.log('start');
    await boss.start();

    console.log('stop');
    await boss.stop();
})();

If I pass noScheduling: true, it will properly exit, however, as soon as I do anything with pg-boss (like call getQueueSize for example, it will hang again, even with noScheduling set:

const PgBoss = require('pg-boss');

(async function main() {
    boss = new PgBoss({
        host: 'localhost',
        port: 5432,
        database: 'postgres',
        user: 'postgres',
        retentionMinutes: 30,
        max: 1,
        noScheduling: true,
    });

    console.log('start');
    await boss.start();

    console.log('size', await boss.getQueueSize('crawl'));  // <- comment this to make it break again

    console.log('stop');
    await boss.stop();
})();
romandecker commented 3 years ago

Did some more investigation and figured out what the issue is:

Not sure if I would still consider this a bug... I'll leave it up to you to close ;)

ivank commented 3 years ago

I would definitely echo this as a bug. In normal operations, waiting for 60 seconds might be fine, but when we run in in tests, 60 seconds to wait for the test is not something we'd like to do. We are still on the last major version of pgboss (before the introduction of timekeep) because of this. Maybe there is a way to start pgboss without timekeep in tests?

romandecker commented 3 years ago

There is actually, we pass these options to PgBoss in our tests:

{
  noSupervisor: true,
  noScheduling: true
}
ivank commented 3 years ago

Thanks @romandecker that pretty much solved our problems. Would love this to be documented a bit more prominently as it is the first thing you encounter as a problem when you write a test.