josegonzalez
commented
1 month ago Are you running the import/export over ssh?
Open chapterjason opened 1 month ago
josegonzalez
commented
1 month ago Are you running the import/export over ssh?
chapterjason
commented
1 month ago @josegonzalez Yes, they are running with the ~/.dokku/contrib/dokku_client.sh-Client.
chapterjason
commented
1 month ago Same result running as root on server:
root@server/home/user # dokku redis:create rtest
Waiting for container to be ready
=====> Redis container created: rtest
=====> rtest redis service information
Config dir: /var/lib/dokku/services/redis/rtest/config
Config options:
Data dir: /var/lib/dokku/services/redis/rtest/data
Dsn: redis://...@dokku-redis-rtest:6379
Exposed ports: -
Id: f638fcf6513216ce0cecd667847ba6f854a21f2ac8fb8726c70a633ac69911fe
Internal ip: 172.17.0.2
Initial network:
Links: -
Post create network:
Post start network:
Service root: /var/lib/dokku/services/redis/rtest
Status: running
Version: redis:7.2.5
root@server/home/user # dokku redis:export rtest > redis.dump
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
root@server/home/user # dokku redis:import rtest < redis.dump
/var/lib/dokku/plugins/enabled/redis/functions: line 187: /var/lib/dokku/services/redis/rtest/data/dump.rdb: Permission denied
root@server/home/user #Other than that warning - which we should fix - seems fine here:
root@dokku:~# dokku redis:create rtest
Waiting for container to be ready
=====> Redis container created: rtest
=====> rtest redis service information
Config dir: /var/lib/dokku/services/redis/rtest/config
Config options:
Data dir: /var/lib/dokku/services/redis/rtest/data
Dsn: redis://:269c828580fa125b1acffe1a68a18e437d38d8a9709c3ba531148611e8a82ee7@dokku-redis-rtest:6379
Exposed ports: -
Id: ccd4de703c0fe5fe6da322a8a18d1fe18c85a90d745aa59488a92d50a7c9c1f9
Internal ip: 172.17.0.12
Initial network:
Links: -
Post create network:
Post start network:
Service root: /var/lib/dokku/services/redis/rtest
Status: running
Version: redis:7.2.5
root@dokku:~# dokku redis:export rtest > redis.dump
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
root@dokku:~# ls redis.dump
redis.dump
root@dokku:~# ls -lah redis.dump
-rw-r--r-- 1 root root 88 Jul 19 00:19 redis.dump
root@dokku:~# dokku redis:import rtest < redis.dumpCan you run the following and post the output?
ls -lah /var/lib/dokku/services/redis/rtest/dataThe dump.rdb in there might be owned by root for some reason and therefore you don't have access to write to it.
chapterjason
commented
1 month ago root@server~ # dokku redis:logs rtest
1:C 19 Jul 2024 04:53:14.474 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:C 19 Jul 2024 04:53:14.474 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 19 Jul 2024 04:53:14.474 * Redis version=7.2.5, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 19 Jul 2024 04:53:14.474 * Configuration loaded
1:M 19 Jul 2024 04:53:14.474 * monotonic clock: POSIX clock_gettime
1:M 19 Jul 2024 04:53:14.475 # Failed to write PID file: Permission denied
1:M 19 Jul 2024 04:53:14.475 * Running mode=standalone, port=6379.
1:M 19 Jul 2024 04:53:14.475 * Server initialized
1:M 19 Jul 2024 04:53:14.475 * Ready to accept connections tcp
root@server~ # ls -lah redis.dump
-rw-r--r-- 1 root root 0 Jul 19 06:53 redis.dump
root@server~ # dokku redis:import rtest < redis.dump
/var/lib/dokku/plugins/enabled/redis/functions: line 187: /var/lib/dokku/services/redis/rtest/data/dump.rdb: Permission denied
root@server~ # dokku redis:logs rtest
! Service container does not existroot@server~ # ls -lah /var/lib/dokku/services/redis/rtest/data
total 8.0K
drwxr-xr-x 2 999 dokku 4.0K Jul 19 06:53 .
drwxr-xr-x 4 dokku dokku 4.0K Jul 19 06:53 ..The folder is empty.
chapterjason
commented
1 month ago Had the case that there is a file:
drwxr-xr-x 2 999 dokku 4.0K Jul 19 21:51 .
drwxr-xr-x 4 dokku dokku 4.0K Jul 19 06:56 ..
-rw------- 1 999 systemd-journal 88 Jul 19 21:51 dump.rdbReally confusing... 😄
systemd-journal also resolved in 999
chapterjason
commented
1 month ago I think I've fixed it like this:
suppress_output service_container_rm "$SERVICE"
"$DOCKER_BIN" container run --rm -v "$SERVICE_HOST_ROOT/data:/data" "$PLUGIN_IMAGE:$PLUGIN_IMAGE_VERSION" bash -c "rm -f /data/dump.rdb"
+ "$DOCKER_BIN" container run --rm -i -v "$SERVICE_HOST_ROOT/data:/data" "$PLUGIN_IMAGE:$PLUGIN_IMAGE_VERSION" bash -c "cat > /data/dump.rdb" < <(cat)
- cat >"$SERVICE_HOST_ROOT/data/dump.rdb"
"$DOCKER_BIN" container run --rm -v "$SERVICE_HOST_ROOT/data:/data" "$PLUGIN_IMAGE:$PLUGIN_IMAGE_VERSION" bash -c "chown redis: /data/dump.rdb"
suppress_output service_start "$SERVICE"It also makes sense, as we are trying to manipulate a data fs-store outside from the actual used container.
josegonzalez
commented
1 month ago Wait so was there a file inside the container or outside the container?
chapterjason
commented
1 month ago I've never checked inside the container. I just realized that we always interact with the dump file from within temporary or the service container itself, except for the stdin redirect to the file (write operation) is running on the host system.
josegonzalez
commented
1 month ago Sorry I'm asking where this is from: https://github.com/dokku/dokku-redis/issues/248#issuecomment-2240020657
You mentioned previously there was nothing at that directory but now there is? Just trying to figure out whats going on since your comments aren't super clear and I'm unable to replicate this otherwise.
chapterjason
commented
1 month ago Here the state after every command, just splitted for better overview.
Cleanup
root@server ~ # dokku redis:destroy rtest --force
=====> Deleting rtest
=====> Pausing container
Container paused
Removing container
Removing data
=====> Redis container deleted: rtest
root@server ~ # ls -lar /var/lib/dokku/services/redis/rtest/data
ls: cannot access '/var/lib/dokku/services/redis/rtest/data': No such file or directoryCreate
root@server ~ # dokku redis:create rtest
Waiting for container to be ready
=====> Redis container created: rtest
=====> rtest redis service information
Config dir: /var/lib/dokku/services/redis/rtest/config
Config options:
Data dir: /var/lib/dokku/services/redis/rtest/data
Dsn: redis://:[...]@dokku-redis-rtest:6379
Exposed ports: -
Id: fc057791ef92f0bacfceabf9e9bc11cbae1541de583768349c4b01cda21a0160
Internal ip: 172.17.0.7
Initial network:
Links: -
Post create network:
Post start network:
Service root: /var/lib/dokku/services/redis/rtest
Status: running
Version: redis:7.2.5
root@server ~ # ls -lar /var/lib/dokku/services/redis/rtest/data
total 8
drwxr-xr-x 4 dokku dokku 4096 Jul 29 19:58 ..
drwxr-xr-x 2 999 dokku 4096 Jul 29 19:58 .
Export
root@server ~ # dokku redis:export rtest > redis.dump
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
root@server ~ # ls -lar /var/lib/dokku/services/redis/rtest/data
total 12
-rw------- 1 999 systemd-journal 88 Jul 29 19:59 dump.rdb
drwxr-xr-x 4 dokku dokku 4096 Jul 29 19:58 ..
drwxr-xr-x 2 999 dokku 4096 Jul 29 19:59 .
root@server ~ # ls -lar redis.dump
-rw-r--r-- 1 root root 88 Jul 29 19:59 redis.dumpImport
root@server ~ # dokku redis:import rtest < redis.dump
/var/lib/dokku/plugins/enabled/redis/functions: line 190: /var/lib/dokku/services/redis/rtest/data/dump.rdb: Permission denied
root@server ~ # ls -lar /var/lib/dokku/services/redis/rtest/data
total 8
drwxr-xr-x 4 dokku dokku 4096 Jul 29 19:59 ..
drwxr-xr-x 2 999 dokku 4096 Jul 29 20:00 .
Description of problem
Running
redis:importto import previously exported data causes permission error.How reproducible
I've just created the service and exported the data to a file, after that just import it with the commands in streps to reproduce.
Steps to Reproduce
Actual Results
Expected Results
Imports the data without error.
Environment Information
dokku report APP_NAMEoutputThe redis service isn't linked to any application yet.
Here is the output of the command with an app:
How (deb/make) and where (AWS, VirtualBox, physical, etc.) was Dokku installed?:
Using the install script (deb) on a physical server.
Additional information
Latest lines of the command with trace:on:
Permissions of the folder:
The data folder is empty.
After changing the permission with
chown dokku:dokku /var/lib/dokku/services/redis/rtest/datait continues but fails at start of redis; after that the permission will be set back as listed above.Full output