fetchMessages({limit:Infinity}) does not return all messages

[aidulcandra]

Is there an existing issue for this?

• [X] I have searched the existing issues

Describe the bug

When I try to fetch all messages and passing the limit of Infinity, it does not return all the messages. In my case I have 59 messages, but it only returns 21. But then when I passed a large number like 100 it would return all the 59 messages.

Expected behavior

Expected that the fetchMessages function return all the messages to the earliest

Steps to Reproduce the Bug or Issue

Use fetchMessages and pass limit:Infinity, try it with chats with large number of messages.

Relevant Code

No response

Browser Type

Chromium

WhatsApp Account Type

Standard

Does your WhatsApp account have multidevice enabled?

No, I am not using Multi Device

Environment

OS: Windows Phone OS: Android whatsapp-web.js version 1.16.4 WhatApp Web version 2.2208.7 Node.js version 16.13.1

Additional context

No response

[allfii]

I also experienced the same issue

[dsandrade]

I had the same prooblem

[sergiooak]

let allMessages = await chat.fetchMessages({
      limit: 99,
    });

Until last version of WhatsApp, if you fetch more messages than the chat have it just works fine, but since it updated (to me, it was yesterday) it enters an infinity loop, I guess both problem is related

[PurpShell]

guessing this is a rate limit thing, take the messages in incremental bites (like get 50 and delay, and then repeat)

[allfii]

@PurpShell could you show me how to do incremental bites? I only see "limit" option and there is no "offset" option in the fetchMessages function

[PurpShell]

Seems like we should work on pagination or a delay functionality

[satonio]

Same problem. With limit 50 it hangs most of times.

[deepto98]

For me var messages = await chat.fetchMessages({ limit: Number.MAX_VALUE });, loads around 5000 messages in a group chat, but not all. Using Infinity didn't work at all for me.

[jasonnathan]

The workaround will be to manually load up the messages using the browser. Here's what I did:

1. Using the example shell.js in the repository, I created a repl instance on my mac
2. This loads up chrome via puppeteer
3. Painfully scrolled up... and scrolled up... and scrolled up...
4. Back in the repl:

wwebjs> let chats = await client.getChats();
undefined
wwebjs> let chat = chats.find(c => // logic to retrieve the conversation you want)
undefined
wwebjs> let messages = await chat.fetchMessages({limit: 1e4}) // 10000 messages
undefined

5. Now you'd have all the messages you loaded via the browser. Painful but... if you're lazy like I am, create this functionality beforehand and rerun it:

const rerun = (client) => {
  let myChat = null
  let messages = []
  const getMyChat = async (findFn) => {
    const chats = await client.getChats()
    myChat = chats.find(findFn)
  }
  const fetchMessages = async (limit = Infinity) => {
    if(myChat)  {
      messages = await myChat.fetchMessages({limit})
    }
    return Promise.resolve(messages) // in case we didn't get into the if block
  }
  return async () => {
    await getMyChat(chat => chat.name === "DoNotMessageManually")
    await fetchMessages()
    console.log(`Retrieved ${messages.length} messages`)
    return messages
  }
}

6. Then bind it to repl's context:

client.on('ready', () => {
  const shell = repl.start('wwebjs> ');
  shell.context.client = client;
  shell.context.runner = rerun(client)
  shell.on('exit', async () => {
      await client.destroy();
  });
});

7. Now after the manual labour of scrolling endlessly in the browser...

wwebjs> await runner()
Retrieved 3123 messages

[icheered]

@jasonnathan Did you scroll up manually or is it scriptable?