Closed slatinsky closed 1 year ago
Thanks!
- Old bug - timestamp supplied using
--before
can overflow into the future - for example--before "2010-08-30T00:00:00.000Z"
overflows to snowflake16548918821562351616
(2140-01-12T07:35:11.104Z
). By adding more early checks, there may be more cases, where this bug can be an issue
Can you please also make a PR for that in Snowflake.FromDate(...)
method (or wherever applicable)? We can assume that a valid snowflake should be at least after Discord was launched.
By adding new threads support,
Fetching channels...
stage takes too long for bigger guilds with many threads. Some data doesn't need to be fetched, because we know, that those channels/threads won't be exported.If
--after
parameter is specified, the main speed-up is to break early of thread fetching, by sorting threads byLastMessageId
(endpointchannels/{channel.Id}/threads/search
- query parameterssort_by=last_message_time&sort_order=desc
) and comparingAfter
andLastMessageId
. This check is implemented only for user tokensSkip thread export for channel if:
channel.Kind == ChannelKind.GuildCategory
- categories never have threadschannel.Kind.IsVoice()
- voice channels never have threadschannel.LastMessageId
- if the channel is empty, the channel doesn't have threads (if the channel is forum post,LastMessageId
will bechannel.Id
of the last forum post)--before
)Before < channel.Id
- if the channel is created in the futureSkip export of these ordinary channels if:
After > channel.LastMessageId
- the channel doesn't have new messages (BUT it can have new messages in threads, that's why we don't skip it in thread exports)Potential problems:
Failed to export XXX channel(s):
--before
can overflow into the future - for example--before "2010-08-30T00:00:00.000Z"
overflows to snowflake16548918821562351616
(2140-01-12T07:35:11.104Z
). By adding more early checks, there may be more cases, where this bug can be an issueTested - CLI json export using
exportGuild
with--include-threads All
and combinations of--before
and--after
parameters