SOHU-Co / kafka-node

Node.js client for Apache Kafka 0.8 and later.
MIT License
2.66k stars 628 forks source link

how to commit message manually? #1097

Open dearpeach opened 5 years ago

dearpeach commented 5 years ago

I don't want to set autoCommit, I want to commit message manually?

My code is:

var kafka = require('kafka-node'),
    Consumer = kafka.Consumer,
    client = new kafka.Client(),
    consumer = new Consumer(
        client,
        [
            { topic: 'test11' }
        ],
        {
            // groupId: "web-caesar-eye",
            autoCommit: false
        }
    );

consumer.on('message', function (message) {
    console.log(message);

    consumer.commit((err, data) => {
        console.log(err, data)
    })
});

consumer.on('error', function(error) {
    console.log('error: ', error)
})

but it doesn't work, kafka still send data again

chrisandrews7 commented 5 years ago

@dearpeach the reason you can't commit is because the commitInterval window is still enforced. When the consumer first connects it commits, and because the autoCommitIntervalMs default value is 5000, your manual commits are ignored until that time has elapsed since the previous commit.

You can use a force commit, by setting the first argument to true. This will ignore the commit interval.

consumer.commit(true, (err, data) => {
    console.log(err, data);
})