`timescale/timescaledb` IMAGE stuck at `restarting` status

[mewben]

Hi, I'm trying to use a custom image from timescale/timescaledb through the following:

export POSTGRES_IMAGE="timescale/timescaledb"
export POSTGRES_IMAGE_VERSION="latest-pg10"

dokku postgres:create testdb

       Waiting for container to be ready
       Creating container database
       Securing connection to database
=====> Postgres container created: testdb
=====> Container Information
       Config dir:          /var/lib/dokku/services/postgres/testdb/config
       Data dir:            /var/lib/dokku/services/postgres/testdb/data
       Dsn:                 postgres://postgres:c5b9973c99d0482d1e4d5975778c6e60@dokku-postgres-testdb:5432/testdb
       Exposed ports:       -
       Id:                  27f10adf1b0aa554a5019af5868a36fdb980aac7b6726ec946771414a2114c45
       Internal ip:
       Links:               -
       Service root:        /var/lib/dokku/services/postgres/testdb
       Status:              running
       Version:             timescale/timescaledb:latest-pg10

But when I do dokku postgres:list

NAME          VERSION                             STATUS      EXPOSED PORTS  LINKS
testdb        timescale/timescaledb:latest-pg10   restarting  -              -

I tried restarting/stopping/removing the testdb but it throws an error like this:

 !     Service is already stopped
=====> Starting container
docker: Error response from daemon: Conflict. The container name "/dokku.postgres.testdb" is already in use by container "27f10adf1b0aa554a5019af5868a36fdb980aac7b6726ec946771414a2114c45". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.

I also tried restarting the os but still no luck. :(

I noticed that there's no Internal ip in the container. Maybe this is the reason why it's stuck at restarting? Please help me what to do. Thank you very much.

[josegonzalez]

Apologies for the late response here.

I think the service is restarting for an unrelated reason. As its a custom image, we can't really provide a ton of support. I would personally do something like docker logs testdb to see if there is any error message coming from the container.

Hope that helps in the future.

[sec0ndhand]

I'm not trying to reopen this issue, but help document for anyone else who stumbles upon this.

The issue with timescaledb is it appears their image deletes out the generated certs. I copied a cert from another postgres instance in dokku to /var/lib/dokku/services/postgres/<DB_NAME>/data per the docs in the Security section, and it seemed to resolve the rebooting issue.

[nileio]

I encountered the same issue and was fixed by copying both server.crt and server.key from a working postgres instance to /var/lib/dokku/services/postgres/<DB_NAME>/data as suggested by @sec0ndhand .. Note you need to sudo -i to be root in order to copy those files. May I ask if it is possibly something that dokku internally does before starting the server ? that is does those certificate files get generated by dokku ? can we have this issue open in order to find the root cause ?

[wonderer007]

@nileio @sec0ndhand @josegonzalez I am having exactly same issue. But the error log is

2022-03-24 01:58:42.304 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:43.305 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:44.306 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:45.307 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:46.309 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:47.310 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:48.311 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:49.312 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:50.313 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:51.314 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:52.315 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:53.317 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:54.318 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:55.319 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:56.320 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:57.321 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:58.322 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:58:59.323 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:00.324 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:01.326 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:02.327 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:03.328 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:04.329 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:05.330 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:06.331 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:07.332 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:08.334 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:09.335 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:10.336 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:11.337 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:12.338 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:13.339 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:14.340 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:15.341 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:16.343 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:17.344 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:18.345 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:19.346 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:20.347 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:21.348 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:22.349 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:23.350 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:24.352 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:25.353 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:26.354 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:27.355 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:28.356 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:29.357 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:30.358 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:31.359 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:32.361 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:33.362 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:34.363 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:35.364 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:36.365 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory
2022-03-24 01:59:37.366 UTC [29] ERROR:  could not open directory "pg_logical/snapshots": No such file or directory

I tried the above solution copying the server.key and server.crt files but didn't work.

Can I recover data from /var/lib/dokku/services/postgres/<DB_NAME>/data directory ?

@josegonzalez can you please reopen the issue.

[josegonzalez]

@wonderer007 if your error is different and the solution above isnt working, I really doubt its the same issue. Additionally, this is now a years old ticket.

Please file a new ticket with all the information asked for.