Can't load lots of messages with user token

[idanyas]

I have a discord channel with ~248.9gb of data and I use a user token, the problem is that at first everything worked, but when the data size increased I started getting:

discordFS >>> booting discordFS +0ms (and then ddrive closes)

The Discord account itself is alive, through the client I can easily read messages and download files from the channel. (as well as upload them)

[idanyas]

after very many runs it started

[forscht]

Are you trying from local machine? I think it's memory issue

[idanyas]

I don't think the problem is memory, I have 24GB of RAM on my VPS and most of the time memory is free (including when I run ddrive, you can see it in the screenshot below)

I also tried running the bot, and after 5 minutes I got the standard error that the messages in the channel were not sent by the bot. (I think it managed to read all the messages successfully, unlike the user)

[forscht]

Does your bot have Message Content Intent enabled?

[forscht]

@idanyas Can you join https://discord.gg/3TCZRYafhW to discuss further?

[idanyas]

ah, I answer through a translator. I don't think we can talk normally, but I can try.

[idanyas]

Does your bot have Message Content Intent enabled?

Oh, I think I forgot about that. Gave the bot this feature and it was able to start.

[forscht]

Closing the issue and updating the README

[idanyas]

But it still does not start using the user token.

[idanyas]

@forscht

[forscht]

@idanyas Have you tried this option? https://github.com/forscht/ddrive/blob/9979a00bb3c51534ebb0f88c8bb5d4c31d9f67b5/README.md?plain=1#L121

[forscht]

const restOpts = {
    timeout: '60000',
    authPrefix: '', // Uncomment this if you're using user token
}
const discordFS = new DiscordFS({
    token, channelId, rest: restOpts, chunkSize,
})

[idanyas]

Yes, of course.

The thing is that when there was not much data, everything ran smoothly. However, when a lot of data is stored (in my case about 250GB) it stops starting.

and at the same time if I start from the bot (not from the user) everything runs smoothly

[idanyas]

Maybe mark it as a bug?

[forscht]

@idanyas I think it's not bug but Discord is rate limiting get message API for users token

[idanyas]

I think so, too. When you start with a user token it starts instantly unlike bots. (until it reaches rate limits and stops starting, of course.)

I assume for now the solution is not to store too much data using the user token. Given the recent update with 500MB files makes it a lot easier.