Error while exporting: Input string was not in a correct format

xero-lib commented 10 months ago

Version

2.42.3

Flavor

CLI (Command-Line Interface)

Platform

Arch Linux 6.5.9

Export format

JSON

Steps to reproduce

Unsure, seems to mainly affect accounts in large numbers of servers, so it could be related to numerous things. Potentially related to #1103, but results in a different error.

Details

Command:

dotnet run exportall --token "<token>" --media --reuse-media
 --format Json --media-dir /mnt/2ext/Backup/DiscordExports/recent/media/ -o /mnt/2ext/Backup/DiscordExports/recent/

Error:

System.FormatException: Input string was not in a correct format.
  at System.DateTimeFormat.GetRealFormat(ReadOnlySpan`1 format, DateTimeFormatInfo dtfi)
  at System.DateTimeFormat.ExpandPredefinedFormat(ReadOnlySpan`1 format, DateTime& dateTime, DateTimeFormatInfo& dtfi, TimeSpan offset)
  at System.DateTimeFormat.FormatStringBuilder(DateTime dateTime, ReadOnlySpan`1 format, DateTimeFormatInfo dtfi, TimeSpan offset, ValueStringBuilder& result)
  at System.DateTimeFormat.Format(DateTime dateTime, String format, IFormatProvider provider, TimeSpan offset)
  at DiscordChatExporter.Core.Exporting.ExportContext.FormatDate(DateTimeOffset instant, String format) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Exporting/ExportContext.cs:41
  at DiscordChatExporter.Core.Exporting.PlainTextMarkdownVisitor.VisitTimestampAsync(TimestampNode timestamp, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Exporting/PlainTextMarkdownVisitor.cs:92
  at DiscordChatExporter.Core.Markdown.Parsing.MarkdownVisitor.VisitAsync(MarkdownNode node, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Markdown/Parsing/MarkdownVisitor.cs:132
  at DiscordChatExporter.Core.Markdown.Parsing.MarkdownVisitor.VisitAsync(IEnumerable`1 nodes, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Markdown/Parsing/MarkdownVisitor.cs:145
  at DiscordChatExporter.Core.Exporting.PlainTextMarkdownVisitor.FormatAsync(ExportContext context, String markdown, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Exporting/PlainTextMarkdownVisitor.cs:113
  at DiscordChatExporter.Core.Exporting.JsonMessageWriter.FormatMarkdownAsync(String markdown, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Exporting/JsonMessageWriter.cs:40
  at DiscordChatExporter.Core.Exporting.JsonMessageWriter.WriteMessageAsync(Message message, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Exporting/JsonMessageWriter.cs:372
  at DiscordChatExporter.Core.Exporting.MessageExporter.ExportMessageAsync(Message message, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Exporting/MessageExporter.cs:76
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:79
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:64
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:85
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_2.<<ExportAsync>b__2>d.MoveNext() in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:228
  at DiscordChatExporter.Cli.Utils.Extensions.ConsoleExtensions.StartTaskAsync(ProgressContext context, String description, Func`2 performOperationAsync) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Cli/Utils/Extensions/ConsoleExtensions.cs:53
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_1.<<ExportAsync>b__1>d.MoveNext() in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:202
  at System.Threading.Tasks.Parallel.<>c__50`1.<<ForEachAsync>b__50_0>d.MoveNext()
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_0.<<ExportAsync>b__0>d.MoveNext() in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:191
  at Spectre.Console.Progress.<>c__DisplayClass27_0.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:98
  at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:150
  at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in /_/src/Spectre.Console/Internal/DefaultExclusivityMode.cs:46
  at Spectre.Console.Progress.StartAsync[T](Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:151
  at Spectre.Console.Progress.StartAsync(Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:96
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.ExportAsync(IConsole console, IReadOnlyList`1 channels) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:181
  at DiscordChatExporter.Cli.Commands.ExportAllCommand.ExecuteAsync(IConsole console) in /home/User/Apps/DiscordChatExporter/DiscordChatExporter.Cli/Commands/ExportAllCommand.cs:207
  at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in /_/CliFx/CliApplication.cs:153
  at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in /_/CliFx/CliApplication.cs:193

Checklist

[X] I have looked through existing issues to make sure that this bug has not been reported before
[X] I have provided a descriptive title for this issue
[X] I have made sure that that this bug is reproducible on the latest version of the application
[X] I have provided all the information needed to reproduce this bug as efficiently as possible
[ ] I have sponsored this project

Tyrrrz commented 9 months ago

This seems to be caused by a weird timestamp format, in a <t:...>. Can you test if the same export command works through Docker, to isolate if it's a DCE issue or something caused by missing dependencies?

xero-lib commented 9 months ago

This seems to be caused by a weird timestamp format, in a <t:...>. Can you test if the same export command works through Docker, to isolate if it's a DCE issue or something caused by missing dependencies?

Exporting with Docker now, will provide updates as they happen.

xero-lib commented 9 months ago

I get the same error in Docker.

Command:

docker run --rm -it -v$PWD:/out tyrrrz/discordchatexporter exportall --include-dm false --token "<token>" --media --reuse-media --parallel 3 --format Json --media-dir ./media

Output:

ERROR
System.FormatException: Input string was not in a correct format.
  at System.DateTimeFormat.ExpandStandardFormatToCustomPattern(Char format, DateTimeFormatInfo dtfi)
  at System.DateTimeFormat.Format(DateTime dateTime, String format, IFormatProvider provider, TimeSpan offset)
  at DiscordChatExporter.Core.Exporting.PlainTextMarkdownVisitor.VisitTimestampAsync(TimestampNode timestamp, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/PlainTextMarkdownVisitor.cs:92
  at DiscordChatExporter.Core.Markdown.Parsing.MarkdownVisitor.VisitAsync(MarkdownNode node, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Markdown/Parsing/MarkdownVisitor.cs:130
  at DiscordChatExporter.Core.Markdown.Parsing.MarkdownVisitor.VisitAsync(IEnumerable`1 nodes, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Markdown/Parsing/MarkdownVisitor.cs:145
  at DiscordChatExporter.Core.Exporting.PlainTextMarkdownVisitor.FormatAsync(ExportContext context, String markdown, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/PlainTextMarkdownVisitor.cs:113
  at DiscordChatExporter.Core.Exporting.JsonMessageWriter.FormatMarkdownAsync(String markdown, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/JsonMessageWriter.cs:40
  at DiscordChatExporter.Core.Exporting.JsonMessageWriter.WriteMessageAsync(Message message, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/JsonMessageWriter.cs:372
  at DiscordChatExporter.Core.Exporting.MessageExporter.ExportMessageAsync(Message message, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/MessageExporter.cs:76
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:79
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:64
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:85
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_2.<<ExportAsync>b__2>d.MoveNext() in /tmp/dce/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:227
  at DiscordChatExporter.Cli.Utils.Extensions.ConsoleExtensions.StartTaskAsync(ProgressContext context, String description, Func`2 performOperationAsync) in /tmp/dce/DiscordChatExporter.Cli/Utils/Extensions/ConsoleExtensions.cs:53
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_1.<<ExportAsync>b__1>d.MoveNext() in /tmp/dce/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:201
  at System.Threading.Tasks.Parallel.<>c__53`1.<<ForEachAsync>b__53_0>d.MoveNext()
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_0.<<ExportAsync>b__0>d.MoveNext() in /tmp/dce/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:190
  at Spectre.Console.Progress.<>c__DisplayClass27_0.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:98
  at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext()
  at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func)
  at Spectre.Console.Progress.StartAsync[T](Func`2 action)
  at Spectre.Console.Progress.StartAsync(Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:96
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.ExportAsync(IConsole console, IReadOnlyList`1 channels) in /tmp/dce/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:180
  at DiscordChatExporter.Cli.Commands.ExportAllCommand.ExecuteAsync(IConsole console) in /tmp/dce/DiscordChatExporter.Cli/Commands/ExportAllCommand.cs:207
  at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in /_/CliFx/CliApplication.cs:153
  at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in /_/CliFx/CliApplication.cs:193

xero-lib commented 9 months ago

I also got a few other errors along the way, such as:

ERROR
System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
  at System.Net.Http.HttpConnection.FillAsync(Boolean async)
  at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.ReadAsyncCore(Memory`1 buffer, CancellationToken cancellationToken)
  at System.Text.Json.JsonDocument.ReadToEndAsync(Stream stream, CancellationToken cancellationToken)
  at System.Text.Json.JsonDocument.ParseAsyncCore(Stream utf8Json, JsonDocumentOptions options, CancellationToken cancellationToken)
  at JsonExtensions.Http.HttpExtensions.ReadAsJsonAsync(HttpContent httpContent, CancellationToken cancellationToken) in /_/JsonExtensions/Http/HttpExtensions.cs:22
  at DiscordChatExporter.Core.Discord.DiscordClient.GetJsonResponseAsync(String url, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Discord/DiscordClient.cs:171
  at DiscordChatExporter.Core.Discord.DiscordClient.GetMessagesAsync(Snowflake channelId, Nullable`1 after, Nullable`1 before, IProgress`1 progress, CancellationToken cancellationToken)+MoveNext() in /tmp/dce/DiscordChatExporter.Core/Discord/DiscordClient.cs:596
  at DiscordChatExporter.Core.Discord.DiscordClient.GetMessagesAsync(Snowflake channelId, Nullable`1 after, Nullable`1 before, IProgress`1 progress, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:64
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:64
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:85
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_2.<<ExportAsync>b__2>d.MoveNext() in /tmp/dce/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:227
  at DiscordChatExporter.Cli.Utils.Extensions.ConsoleExtensions.StartTaskAsync(ProgressContext context, String description, Func`2 performOperationAsync) in /tmp/dce/DiscordChatExporter.Cli/Utils/Extensions/ConsoleExtensions.cs:53
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_1.<<ExportAsync>b__1>d.MoveNext() in /tmp/dce/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:201
  at System.Threading.Tasks.Parallel.<>c__53`1.<<ForEachAsync>b__53_0>d.MoveNext()
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_0.<<ExportAsync>b__0>d.MoveNext() in /tmp/dce/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:190
  at Spectre.Console.Progress.<>c__DisplayClass27_0.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:98
  at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext()
  at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func)
  at Spectre.Console.Progress.StartAsync[T](Func`2 action)
  at Spectre.Console.Progress.StartAsync(Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:96
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.ExportAsync(IConsole console, IReadOnlyList`1 channels) in /tmp/dce/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:180
  at DiscordChatExporter.Cli.Commands.ExportAllCommand.ExecuteAsync(IConsole console) in /tmp/dce/DiscordChatExporter.Cli/Commands/ExportAllCommand.cs:207
  at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in /_/CliFx/CliApplication.cs:153
  at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in /_/CliFx/CliApplication.cs:193

and of course, a 503 which seems like it might be able to be handled better than just exiting.

  ERROR
System.Net.Http.HttpRequestException: Resource temporarily unavailable (discord.com:443)
    System.Net.Sockets.SocketException: Resource temporarily unavailable
      at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
      at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
      at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
      at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
  at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
  at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
  at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
  at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
  at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
  at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
  at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
  at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
  at DiscordChatExporter.Core.Discord.DiscordClient.<>c__DisplayClass4_0.<<GetResponseAsync>b__0>d.MoveNext() in /tmp/dce/DiscordChatExporter.Core/Discord/DiscordClient.cs:50
  at Polly.ResiliencePipeline.<>c__10`1.<<ExecuteAsync>b__10_0>d.MoveNext()
  at Polly.Outcome`1.GetResultOrRethrow()
  at Polly.ResiliencePipeline.ExecuteAsync[TResult](Func`2 callback, CancellationToken cancellationToken)
  at DiscordChatExporter.Core.Discord.DiscordClient.GetResponseAsync(String url, TokenKind tokenKind, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Discord/DiscordClient.cs:36
  at DiscordChatExporter.Core.Discord.DiscordClient.ResolveTokenKindAsync(CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Discord/DiscordClient.cs:101
  at DiscordChatExporter.Core.Discord.DiscordClient.GetResponseAsync(String url, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Discord/DiscordClient.cs:127
  at DiscordChatExporter.Core.Discord.DiscordClient.GetJsonResponseAsync(String url, CancellationToken cancellationToken) in /tmp/dce/DiscordChatExporter.Core/Discord/DiscordClient.cs:138
  at DiscordChatExporter.Core.Discord.DiscordClient.GetGuildChannelsAsync(Snowflake guildId, CancellationToken cancellationToken)+MoveNext() in /tmp/dce/DiscordChatExporter.Core/Discord/DiscordClient.cs:253
  at DiscordChatExporter.Core.Discord.DiscordClient.GetGuildChannelsAsync(Snowflake guildId, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
  at DiscordChatExporter.Cli.Commands.ExportAllCommand.<>c__DisplayClass20_1.<<ExecuteAsync>b__5>d.MoveNext() in /tmp/dce/DiscordChatExporter.Cli/Commands/ExportAllCommand.cs:69
  at DiscordChatExporter.Cli.Commands.ExportAllCommand.<>c__DisplayClass20_1.<<ExecuteAsync>b__5>d.MoveNext() in /tmp/dce/DiscordChatExporter.Cli/Commands/ExportAllCommand.cs:69
  at Spectre.Console.Status.<>c__DisplayClass16_0.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Status/Status.cs:79
  at Spectre.Console.Status.<>c__DisplayClass17_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Status/Status.cs:120
  at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:133
  at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in /_/src/Spectre.Console/Internal/DefaultExclusivityMode.cs:40
  at Spectre.Console.Progress.StartAsync[T](Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:116
  at Spectre.Console.Status.StartAsync[T](String status, Func`2 func) in /_/src/Spectre.Console/Live/Status/Status.cs:117
  at Spectre.Console.Status.StartAsync(String status, Func`2 action) in /_/src/Spectre.Console/Live/Status/Status.cs:77
  at DiscordChatExporter.Cli.Commands.ExportAllCommand.ExecuteAsync(IConsole console) in /tmp/dce/DiscordChatExporter.Cli/Commands/ExportAllCommand.cs:62
  at DiscordChatExporter.Cli.Commands.ExportAllCommand.ExecuteAsync(IConsole console) in /tmp/dce/DiscordChatExporter.Cli/Commands/ExportAllCommand.cs:54
  at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in /_/CliFx/CliApplication.cs:153
  at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in /_/CliFx/CliApplication.cs:193

Tyrrrz commented 9 months ago

@xero-lib try running the export again using the :latest tag (you might need to pull the image again), I added more context to the exception messages. It should indicate which exact message caused the error.

Tyrrrz commented 9 months ago

and of course, a 443

443 is not an error, but the port number.

Both the last two errors indicate that you're having connectivity issues (or Discord is). Such issues are already retried up until a point.

xero-lib commented 9 months ago

and of course, a 443

443 is not an error, but the port number.

Both the last two errors indicate that you're having connectivity issues (or Discord is). Such issues are already retried up until a point.

Sorry, I meant 503, which it's liable not to be.

xero-lib commented 9 months ago

@xero-lib try running the export again using the :latest tag (you might need to pull the image again), I added more context to the exception messages. It should indicate which exact message caused the error.

Will do.

xero-lib commented 9 months ago

Odd, after pulling the latest version, running the exact same command results in the error:

ERROR
DiscordChatExporter.Core.Exceptions.DiscordChatExporterException: Failed to export message #642760494125678659 in channel 'rules' (#381898062269775883).
    System.UnauthorizedAccessException: Access to the path '/out/Discord API - Information - rules [381898062269775883].json' is denied.
        System.IO.IOException: Permission denied
      at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
      at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)
      at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)
      at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
      at System.IO.File.Create(String path)
      at DiscordChatExporter.Core.Exporting.MessageExporter.CreateMessageWriter(String filePath, ExportFormat format, ExportContext context) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:107
      at DiscordChatExporter.Core.Exporting.MessageExporter.GetWriterAsync(CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:59
      at DiscordChatExporter.Core.Exporting.MessageExporter.ExportMessageAsync(Message message, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:70
      at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:81
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:86
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:64
  at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:97
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_2.<<ExportAsync>b__2>d.MoveNext() in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:229
  at DiscordChatExporter.Cli.Utils.Extensions.ConsoleExtensions.StartTaskAsync(ProgressContext context, String description, Func`2 performOperationAsync) in /tmp/app/DiscordChatExporter.Cli/Utils/Extensions/ConsoleExtensions.cs:53
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_1.<<ExportAsync>b__1>d.MoveNext() in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:203
  at System.Threading.Tasks.Parallel.<>c__53`1.<<ForEachAsync>b__53_0>d.MoveNext()
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_0.<<ExportAsync>b__0>d.MoveNext() in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:192
  at Spectre.Console.Progress.<>c__DisplayClass27_0.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:98
  at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:133
  at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in /_/src/Spectre.Console/Internal/DefaultExclusivityMode.cs:40
  at Spectre.Console.Progress.StartAsync[T](Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:116
  at Spectre.Console.Progress.StartAsync(Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:96
  at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.ExportAsync(IConsole console, IReadOnlyList`1 channels) in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:182
  at DiscordChatExporter.Cli.Commands.ExportAllCommand.ExecuteAsync(IConsole console) in /tmp/app/DiscordChatExporter.Cli/Commands/ExportAllCommand.cs:207
  at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in /_/CliFx/CliApplication.cs:153
  at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in /_/CliFx/CliApplication.cs:193

Running docker inspect shows that WorkingDir is still /out.

Running

docker run -it -v $PWD:/out debian
echo hello > /out/hello.txt

results in $PWD/hello.txt containing hello, so I don't think it's a local permission issue. I'm not sure what I can do about this but I'm going to try building from the dockerfile and rerun.

Tyrrrz commented 9 months ago

Maybe it can't create that directory on the host for some reason? 🤔 What if you pre-create that directory?

xero-lib commented 9 months ago

Maybe it can't create that directory on the host for some reason? 🤔 What if you pre-create that directory?

I tried that, as well as not using a media-dir, but received the same error. I'm continuing to troubleshoot, and will get it figured out.

xero-lib commented 9 months ago

I have found the offending message in the server Discord API with the content:

xero-lib commented 9 months ago

It seems to be an issue potentially with invalid time format strings.

Tyrrrz commented 9 months ago

Should be fixed. Please try the latest build.

Tyrrrz / DiscordChatExporter