Closed jamestmartin closed 8 months ago
Here's the diff between Dockerfile for 2.42.3
and latest
:
2.42.3...b3a1cbf
#diff-018a549112
The most likely issue is the USER
instruction. It changed from this:
RUN adduser --disabled-password --no-create-home dce
USER dce
To this:
USER $APP_UID
Where the $APP_UID
is created in the base image (mcr.microsoft.com/dotnet/runtime-deps:8.0-alpine
) like so:
RUN addgroup \
--gid=$APP_UID \
app \
&& adduser \
--uid=$APP_UID \
--ingroup=app \
--disabled-password \
app
From my limited understanding, both approaches seem to be essentially equivalent. Or am I wrong?
Can Reproduce on latest
, stable
. Thanks @jamestmartin for mentioning that 2.42.3
doesn't have this issue.
Does this still happen on latest
after https://github.com/Tyrrrz/DiscordChatExporter/commit/d4be307fb179529d08d2ebc38ec6ab476bde52cb? Make sure you force-pull the image tag if you had pulled it previously.
Okay, I reverted the changes that most likely have affected this. Still not sure exactly why though. Please try latest
again and let me know if it works.
Thanks for the tip to force pull, I still see the error
@Tyrrrz Can reproduce on latest
sudo docker run --rm -it -v /home/me/test:/out tyrrrz/discordchatexporter:latest export -t "***" -c *** -f Json
System.UnauthorizedAccessException: Access to the path '/out/***.json' is denied.
System.IO.IOException: Permission denied
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, I
nt64& fileLength, UnixFileMode& filePermissions, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)
at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
at System.IO.File.Create(String path)
at DiscordChatExporter.Core.Exporting.MessageExporter.CreateMessageWriter(String filePath, ExportFormat format, ExportContext context) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporte
r.cs:105
at DiscordChatExporter.Core.Exporting.MessageExporter.GetWriterAsync(CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:57
at DiscordChatExporter.Core.Exporting.MessageExporter.ExportMessageAsync(Message message, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:68
at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Expo
rting/ChannelExporter.cs:87
at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exportin
g/ChannelExporter.cs:92
at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:70
at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:105
at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_2.<<ExportAsync>b__2>d.MoveNext() in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:228
at DiscordChatExporter.Cli.Utils.Extensions.ConsoleExtensions.StartTaskAsync(ProgressContext context, String description, Func`2 performOperationAsync) in /tmp/app/DiscordChatExporter.Cli/Utils/Extensions/ConsoleExtensions.cs:53
at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_1.<<ExportAsync>b__1>d.MoveNext() in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:202
at System.Threading.Tasks.Parallel.<>c__53`1.<<ForEachAsync>b__53_0>d.MoveNext()
at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_0.<<ExportAsync>b__0>d.MoveNext() in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:191
at Spectre.Console.Progress.<>c__DisplayClass31_0.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:103
at Spectre.Console.Progress.<>c__DisplayClass32_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:138
at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in /_/src/Spectre.Console/Internal/DefaultExclusivityMode.cs:40
at Spectre.Console.Progress.StartAsync[T](Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:121
at Spectre.Console.Progress.StartAsync(Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:101
at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.ExportAsync(IConsole console, IReadOnlyList`1 channels) in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:181
at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.ExportAsync(IConsole console, IReadOnlyList`1 channelIds) in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:315
at DiscordChatExporter.Cli.Commands.ExportChannelsCommand.ExecuteAsync(IConsole console) in /tmp/app/DiscordChatExporter.Cli/Commands/ExportChannelsCommand.cs:25
at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in /_/CliFx/CliApplication.cs:163
at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in /_/CliFx/CliApplication.cs:204
@TheOnlyWayUp can you do docker pull tyrrrz/discordchatexporter:latest
first to make sure it's updated to latest? Otherwise, please do docker run ... tyrrrz/discordchatexporter:latest --version
to see if it's the right commit hash.
If I may give my two cents, because it's a classic recurring problem for many images, the pattern other images use is:
You don't have to follow this, but it has the advantage so running your container for the first time sets the permissions the correct way.
With your image the way it's designed atm, what you have to do at the moment is:
mkdir data
chown 1000:1000 data
docker run -it --rm -v ($pwd)/data:/out tyrrrz/discordchatexporter:stable bla bla
On hosts where the current user is already 1000, the chown is not necessary, and because it's what happens with most people you never notice the problem. But then one user has a UID of 1001 and boom, you can this bug and you need the chown.
I suggest you either switch to the pattern above or document that you need to chown to 1000:1000.
@Silex thanks a lot, that was very insightful. Would you be able to PR one of your described solutions?
Following this chat to see if Docker issue gets fixed. I would attempt to fix it but I don't really know anything about Docker :')
@Tyrrrz started a PR, will ping you when it's ready.
@eelegiap @Murat-U-Saglam
Please try :latest
now
I re-ran docker pull tyrrrz/discordchatexporter:latest
and ran into this error:
/opt/app/docker-entrypoint.sh: exec: line 10: ./DiscordChatExporter.Cli: not found
when I ran
docker run --rm \
-v "$PATH_TO_RAW_DATA":/out \
tyrrrz/discordchatexporter:latest exportguild \
-t "$BOT_TOKEN" \
-g "$SERVER_ID" \
-f Json \
--after "$START" \
--before "$END"
My bad, try now @eelegiap (when this build finishes: https://github.com/Tyrrrz/DiscordChatExporter/actions/runs/7731406381)
Seems to be working for me now. Nice work @Tyrrrz!
I updated about 30 minutes ago and it still gives the "System.UnauthorizedAccessException"
error. Could this be because I'm using rootless docker? The files and folders are owned by 1000:1000. I tried setting the permissions to 777 but that didn't fix it. I have some very important stuff I need docker to be rootless for. This error happens with any server.
This is the command I'm using:
docker run --rm -it -v /path/to/discord.com/guilds:/app/out tyrrrz/discordchatexporter:latest exportguild --include-vc --include-threads All -g 'XXX' --token 'XXX' --reuse-media --media --media-dir "/app/out/%G/%c-media/" --output "/app/out/%G/%c.json" --format Json --markdown false
The path was truncated because it was really long and contained private info.
I updated about 30 minutes ago and it still gives the
"System.UnauthorizedAccessException"
error. Could this be because I'm using rootless docker? The files and folders are owned by 1000:1000. I tried setting the permissions to 777 but that didn't fix it. I have some very important stuff I need docker to be rootless for. This error happens with any server.This is the command I'm using:
docker run --rm -it -v /path/to/discord.com/guilds:/app/out tyrrrz/discordchatexporter:latest exportguild --include-vc --include-threads All -g 'XXX' --token 'XXX' --reuse-media --media --media-dir "/app/out/%G/%c-media/" --output "/app/out/%G/%c.json" --format Json --markdown false
The path was truncated because it was really long and contained private info.
I am having the same issue, used sudo with docker with the same issues. It's a mounting error that I get within my vscode projects since if I run the same command outside of my vscode environment it mounts with no problem
@Twi-Hard is your host user ID also 1000:1000?
Otherwise you might try this: https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/Docker.md#unix-permissions-issues
I solved the issue thanks @Tyrrrz . I importantly identified why it's happening.
The volume mount that the docker compose generated to the container was generated via docker compose, I didn't use mkdir
Therefore using ll
I realized it had root owners. Which caused the permission errors therefore use the guidelines stated in https://github.com/Tyrrrz/DiscordChatExporter/blob/master/.docs/Docker.md#unix-permissions-issues to solve the permission error for the directory.
@Twi-Hard is your host user ID also 1000:1000?
❯ id -u $USER && id -g $USER 1000 1000
The folder it's trying to download to already exists because I'm just updating the download. The folder and it's subfolders/files are owned by 1000:1000
@Twi-Hard is your host user ID also 1000:1000?
❯ id -u $USER && id -g $USER 1000 1000
The folder it's trying to download to already exists because I'm just updating the download. The folder and it's subfolders/files are owned by 1000:1000
Can you try exporting to a new directory, just to isolate the issue?
I changed the folder and still got the error.
DiscordChatExporter.Core.Exceptions.DiscordChatExporterException: Failed to export message #XXX in channel 'rules' (#XXX) of guild 'XXX (#XXX)'.
System.UnauthorizedAccessException: Access to the path '/app/out/XXX' is denied.
System.IO.IOException: Permission denied
at System.IO.FileSystem.CreateDirectory(String fullPath, UnixFileMode unixCreateMode)
at System.IO.Directory.CreateDirectory(String path)
at DiscordChatExporter.Core.Exporting.MessageExporter.GetWriterAsync(CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:54
at DiscordChatExporter.Core.Exporting.MessageExporter.ExportMessageAsync(Message message, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/MessageExporter.cs:68
at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:87
at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:92
at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:70
at DiscordChatExporter.Core.Exporting.ChannelExporter.ExportChannelAsync(ExportRequest request, IProgress`1 progress, CancellationToken cancellationToken) in /tmp/app/DiscordChatExporter.Core/Exporting/ChannelExporter.cs:105
at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_2.<<ExportAsync>b__2>d.MoveNext() in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:231
at DiscordChatExporter.Cli.Utils.Extensions.ConsoleExtensions.StartTaskAsync(ProgressContext context, String description, Func`2 performOperationAsync) in /tmp/app/DiscordChatExporter.Cli/Utils/Extensions/ConsoleExtensions.cs:53
at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_1.<<ExportAsync>b__1>d.MoveNext() in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:205
at System.Threading.Tasks.Parallel.<>c__53`1.<<ForEachAsync>b__53_0>d.MoveNext()
at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.<>c__DisplayClass63_0.<<ExportAsync>b__0>d.MoveNext() in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:194
at Spectre.Console.Progress.<>c__DisplayClass31_0.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:103
at Spectre.Console.Progress.<>c__DisplayClass32_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:138
at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in /_/src/Spectre.Console/Internal/DefaultExclusivityMode.cs:40
at Spectre.Console.Progress.StartAsync[T](Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:121
at Spectre.Console.Progress.StartAsync(Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:101
at DiscordChatExporter.Cli.Commands.Base.ExportCommandBase.ExportAsync(IConsole console, IReadOnlyList`1 channels) in /tmp/app/DiscordChatExporter.Cli/Commands/Base/ExportCommandBase.cs:184
at DiscordChatExporter.Cli.Commands.ExportGuildCommand.ExecuteAsync(IConsole console) in /tmp/app/DiscordChatExporter.Cli/Commands/ExportGuildCommand.cs:103
at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in /_/CliFx/CliApplication.cs:163
at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in /_/CliFx/CliApplication.cs:204
@Twi-Hard the folder is /out
, not /app/out
.
@Twi-Hard the folder is /out, not /app/out.
Thanks :) It works now. I don't understand how I got that wrong since I've only copy/pasted the command from my history since I started using docker a long time ago and it used to work.
Version
2.42.4
Flavor
CLI (Command-Line Interface)
Platform
Docker / Debian
Export format
HTML
Steps to reproduce
Tested and replicated in a fresh Debian VM. I bisected the issue and found that it was introduced in 2.42.4; 2.42.3 was the last working version. The issue is still present in
latest
as of today.Details
Expected behavior: Exports the channel.
Actual behavior:
Based on the version in which this bug was introduced, it is most likely unrelated to #978.
Checklist