Tyrrrz / DiscordChatExporter

Exports Discord chat logs to a file
MIT License
7.48k stars 682 forks source link

Add progress to fetching channels step #1131

Closed slatinsky closed 11 months ago

slatinsky commented 12 months ago

Fetching channels... step takes a while after thread export option (--include-threads) was introduced. The process may seem to be stuck to the user.

This pull request adds a simple status information to exportguild and exportall commands.

exportguild example:

$ DiscordChatExporter.Cli exportguild --guild <guildId> --include-threads All --token <token>
Fetching channels...
  Found 21 channels.
Fetching threads...
  Found 405 threads.
Exporting 426 channel(s)...

Voice Channels / General     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
Text / moderator-only        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
IMPORTANT / 📌-read-me-first ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
IMPORTANT / 📢-announcements ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0%

exportall example:

exportall is similar to exportguild but it prints the guild name

$ DiscordChatExporter.Cli exportall --include-threads All --token <token>
Fetching channels...  (Direct Messages)
  Found 0 channels.
Fetching threads...   (Direct Messages)
  Found 0 threads.
Fetching channels...  (Testing guild)
  Found 11 channels.
Fetching threads...   (Testing guild)
  Found 58 threads.
Exporting 69 channel(s)...

Text Channels / general  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
Voice Channels / General ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0%

Both commands show last parent channel name + thread name while in the process of fetching threads:

...
Fetching threads...
  Found 1 threads.  general - My first thread
Tyrrrz commented 12 months ago

Probably can optimize this too 😁

image

I'll do it

slatinsky commented 12 months ago

Suggestion: instead of tracking progress separately, can just do Found {channels.Count(c => c.IsThread)} threads. Slower, but would avoid extra business logic (i.e. off-by-one errors are likely here) and wouldn't matter on top of already heavy I/O requests anyway.

I agree

What's the PadRight/Substring for? Not super clear.

Let's assume, that the first message is

Found 1 threads. general - My first thread

and the second message is

Found 2 threads. general - hello

if we don't use PadRight/Substring, then the second message would look like this in the terminal:

Found 2 threads. general - hellorst thread

PadRight pads the string with spaces to the right, so that the second message would always overwrite the first one. Substring makes sure that it never overflows.

Also, we print one progress message for all channels, but multiple messages for threads?

Yes, because to get main channels, we need only one request. But to get threads, we need to make multiple request (at least one per channel)

Other notes:

If we could somehow get currently processed channel index and channels.Count from Discord.GetGuildThreadsAsync method, we could print more fancy progress bar.

progress: channelIndex / channels.Count * 100

(for this we would also need to merge the loop foreach (var channel in channels) in DiscordClient.cs to process active and archived threads at the same time)

Tyrrrz commented 12 months ago

@slatinsky could we maybe use the Status display from Spectre.Console? We already use this package for progress reporting.

https://spectreconsole.net/live/status

This could clean up all the padright stuff.

slatinsky commented 11 months ago

in the latest changes, I have added your suggestions to the code and rewritten it to use status from Spectre.Console. Also latest changes from master branch were merged to prevent merge conflicts

Tested manually - cli, exportall and exportguild

Tyrrrz commented 11 months ago

Thank you!