shlinkio / shlink

The definitive self-hosted URL shortener
https://shlink.io
MIT License
3.16k stars 254 forks source link

Db migration timing out using container with Shlink 3.6.x #1819

Closed RealFrancisHe closed 1 year ago

RealFrancisHe commented 1 year ago

How Shlink is set up

Summary

When trying to run a new container on a previously used Postgresql database, the db:migrate times out at 60 seconds. This specifically only applies to the running it using an image, as I was able to originally get Shlink running via the installer, which lags when running the update, but does not timeout.

Command:

docker run \
    --name shlink2 \
    --platform linux/arm64 \
    -p 8080:8080 \
    -e DEFAULT_DOMAIN=xyz.com \
    -e IS_HTTPS_ENABLED=true \ 
    -e GEOLITE_LICENSE_KEY=random_key \
    -e DB_DRIVER=postgres \
    -e DB_NAME=shlink_v3 \ 
    -e DB_USER=user \
    -e DB_PASSWORD=password \
    -e DB_HOST=some_host \
    -e SHELL_VERBOSITY=3 \
    shlinkio/shlink:latest

Logs:

Initializing database if needed... [Running "/usr/local/bin/php bin/cli db:create"]   RUN  '/usr/local/bin/php' 'bin/cli' 'db:create'
  OUT  
  OUT   [OK] Database already exists. Run "db:migrate" command to make sure it is up to
  OUT        date.                                                                     
  OUT  
  OUT  
  RES  Command ran successfully
 Success!
Updating database... [Running "/usr/local/bin/php bin/cli db:migrate"]   RUN  '/usr/local/bin/php' 'bin/cli' 'db:migrate'
  OUT  Migrating database...
  OUT  
  ERR    RUN  '/usr/local/bin/php' 'vendor/doctrine/migrations/bin/doctrine-migrations.php' 'migrations:migrate' '--no-interaction'
  ERR  
In Process.php line 1154:

  [Symfony\Component\Process\Exception\ProcessTimedOutException]               
  The process "'/usr/local/bin/php' 'bin/cli' 'db:migrate'" exceeded the time  
  out of 60 seconds.                                                           

Exception trace:
  at /etc/shlink/vendor/symfony/process/Process.php:1154
 Symfony\Component\Process\Process->checkTimeout() at /etc/shlink/vendor/symfony/process/Process.php:423
 Symfony\Component\Process\Process->wait() at /etc/shlink/vendor/symfony/process/Process.php:249
 Symfony\Component\Process\Process->run() at /etc/shlink/vendor/symfony/console/Helper/ProcessHelper.php:69
 Symfony\Component\Console\Helper\ProcessHelper->run() at /etc/shlink/vendor/shlinkio/shlink-installer/src/Service/InstallationCommandsRunner.php:57
 Shlinkio\Shlink\Installer\Service\InstallationCommandsRunner->execPhpCommand() at /etc/shlink/vendor/shlinkio/shlink-installer/src/Command/InitCommand.php:64
 Shlinkio\Shlink\Installer\Command\InitCommand->Shlinkio\Shlink\Installer\Command\{closure}() at /etc/shlink/vendor/lstrojny/functional-php/src/Functional/Every.php:34
 Functional\every() at /etc/shlink/vendor/shlinkio/shlink-installer/src/Command/InitCommand.php:62
 Shlinkio\Shlink\Installer\Command\InitCommand->execute() at /etc/shlink/vendor/symfony/console/Command/Command.php:312
 Symfony\Component\Console\Command\Command->run() at /etc/shlink/vendor/symfony/console/Application.php:1022
 Symfony\Component\Console\Application->doRunCommand() at /etc/shlink/vendor/symfony/console/Application.php:314
 Symfony\Component\Console\Application->doRun() at /etc/shlink/vendor/symfony/console/Application.php:168
 Symfony\Component\Console\Application->run() at /etc/shlink/vendor/shlinkio/shlink-installer/bin/shlink-installer:13
 include() at /etc/shlink/vendor/bin/shlink-installer:120

init [--skip-initialize-db] [--clear-db-cache] [--initial-api-key] [--download-rr-binary] [--skip-download-geolite]

Thanks! Any help is appreciated.

acelaya commented 1 year ago

From what Shlink version are you updating?

acelaya commented 1 year ago

From what Shlink version are you updating?

I'm asking this because the timeout is probably due to some long migrations execution because of a big database (happens with some migrations).

However, v3.6.x has a bug where the proper timeout is not being set when running the migrations, but these versions do not include new migrations themeselves.

A possible workaround would be to update to an earlier version first, and only after migrations have finished successfully, then update to 3.6.3 (you can see more around updates here https://shlink.io/documentation/update-your-instance/)

And to elaborate on the problem, this is how Shlink <3.6 used to run the entry point commands:

flowchart LR
    A[Entry point] --> B(shlink db:create) --> H(...)
    A --> C(shlink db:migrate)
    C -->|10m timeout| D(doctrine migrations:migrate)
    A --> E(...)

And this is how Shlink >=3.6 does it now

flowchart LR
    A[Entry point] --> B(shlink-installer init)
    B --> C(shlink db:create) --> H(...)
    B -->|1m timeout| D(shlink db:migrate)
    D -->|10m timeout| E(doctrine migrations:migrate)
    B --> F(...)

As you can see, there's a new shlink-installer init command wrapping all the entry point operations, which is also used for non-docker setups.

The bug is that it is not defining it's own 10m timeout, resulting in the error you see if migrations take more than 1 minute.