search

shlinkio / shlink

The definitive self-hosted URL shortener
https://shlink.io
MIT License
3.32k stars 271 forks source link

Unable to set `readOnlyRootFilesystem` in Kubernetes deployment #1967

Closed onedr0p closed 10 months ago

onedr0p commented 10 months ago

Shlink version

3.7.2-non-root

PHP version

3.7.2-non-root

How do you serve Shlink

RoadRunner Docker image

Database engine

PostgreSQL

Database version

16.1

Current behavior

I am trying to set a Kubernetes security context on shlink however it doesn't work and falis with the following error.

            securityContext:
              allowPrivilegeEscalation: false
              readOnlyRootFilesystem: true
              capabilities: { drop: ["ALL"] }
│   ERR  In FileNotWritableException.php line 45:                                                                                                                                         │
│   ERR                                                                                                                                                                                   │
│   ERR    [ProxyManager\Exception\FileNotWritableException]                                                                                                                              │
│   ERR    Cannot rename "/tmp/ShlinkProxy__PM__GeoIp2DatabaseReaderGenerated1cc76100a                                                                                                    │
│   ERR    9790b1d2nlMain" to "/etc/shlink/data/proxies/ShlinkProxy__PM__GeoIp2Databas                                                                                                    │
│   ERR    eReaderGenerated1cc76100a9790b1d241b38c9b0c3801c.php": rename(/tmp/ShlinkPr                                                                                                    │
│   ERR    oxy__PM__GeoIp2DatabaseReaderGenerated1cc76100a9790b1d2nlMain,/etc/shlink/d                                                                                                    │
│   ERR    ata/proxies/ShlinkProxy__PM__GeoIp2DatabaseReaderGenerated1cc76100a9790b1d2                                                                                                    │
│   ERR    41b38c9b0c3801c.php): Read-only file system

Expected behavior

The application should not need r/w access to the root filesystem

How to reproduce

Deploy shlink in Kubernetes and set the security context as I mentioned above.

acelaya commented 10 months ago

Hmmm. Looks like some proxies are written to tmp first and then moved to Shlink's folder.

I'll check if this can be configured.

onedr0p commented 10 months ago

If they could stay in /tmp that would be great since I can mount an emptyDir to that location.

acelaya commented 10 months ago

If they could stay in /tmp that would be great since I can mount an emptyDir to that location.

I'm afraid that's not possible (at least without an unreasonable amount of changes). Shlink needs to write a number of temporary/static files at runtime (GeoLite db file, entity proxies, service proxies, filesystem locks, configuration cache files, etc).

However, all files are (or should) be written inside /etc/shlink/data/???, so you should be able to mount /etc/shlink/data instead of /tmp, if that's an option.

Of course, that leaves the fact that there seems to be something writing in /tmp for some reason, which I need to address.

onedr0p commented 10 months ago

Good idea, so I created a emptyDir mount to /etc/shlink/data as well as /tmp however now I get this error:

Initializing database if needed... [Running "/usr/local/bin/php bin/cli db:create"]   RUN  '/usr/local/bin/php' 'bin/cli' 'db:create'
  OUT
  OUT   [OK] Database already exists. Run "db:migrate" command to make sure it is up to
  OUT        date.
  OUT
  OUT
  RES  Command ran successfully
 Success!
Updating database... [Running "/usr/local/bin/php bin/cli db:migrate"]   RUN  '/usr/local/bin/php' 'bin/cli' 'db:migrate'
  OUT  Migrating database...
  OUT
  ERR    RUN  '/usr/local/bin/php' 'vendor/doctrine/migrations/bin/doctrine-migrations.php' 'migrations:migrate' '--no-interaction'
  ERR    ERR
  ERR    ERR  In InvalidDirectory.php line 15:
  ERR    ERR
  ERR    ERR    [Doctrine\Migrations\Finder\Exception\InvalidDirectory]
  ERR    ERR    Cannot load migrations from "data/migrations" because it is not a valid dir
  ERR    ERR    ectory
  ERR    ERR
  ERR    ERR
  ERR    ERR  Exception trace:
  ERR    ERR    at /etc/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Finder/Exception/InvalidDirectory.php:15
  ERR    ERR   Doctrine\Migrations\Finder\Exception\InvalidDirectory::new() at /etc/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Finder/Finder.php:35
  ERR    ERR   Doctrine\Migrations\Finder\Finder->getRealPath() at /etc/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Finder/GlobFinder.php:20
  ERR    ERR   Doctrine\Migrations\Finder\GlobFinder->findMigrations() at /etc/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/FilesystemMigrationsRepository.php:132
  ERR    ERR   Doctrine\Migrations\FilesystemMigrationsRepository->loadMigrationsFromDirectories() at /etc/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/FilesystemMigrationsRepository.php:108
  ERR    ERR   Doctrine\Migrations\FilesystemMigrationsRepository->getMigrations() at /etc/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Tools/Console/Command/MigrateCommand.php:160
  ERR    ERR   Doctrine\Migrations\Tools\Console\Command\MigrateCommand->execute() at /etc/shlink/vendor/symfony/console/Command/Command.php:326
  ERR    ERR   Symfony\Component\Console\Command\Command->run() at /etc/shlink/vendor/symfony/console/Application.php:1078
  ERR    ERR   Symfony\Component\Console\Application->doRunCommand() at /etc/shlink/vendor/symfony/console/Application.php:324
  ERR    ERR   Symfony\Component\Console\Application->doRun() at /etc/shlink/vendor/symfony/console/Application.php:175
  ERR    ERR   Symfony\Component\Console\Application->run() at /etc/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Tools/Console/ConsoleRunner.php:95
  ERR    ERR   Doctrine\Migrations\Tools\Console\ConsoleRunner::run() at /etc/shlink/vendor/doctrine/migrations/bin/doctrine-migrations.php:45
  ERR    ERR   Doctrine\Migrations\{closure}() at /etc/shlink/vendor/doctrine/migrations/bin/doctrine-migrations.php:46
  ERR    ERR
  ERR    ERR  migrations:migrate [--write-sql [WRITE-SQL]] [--dry-run] [--query-time] [--allow-no-migration] [--all-or-nothing [ALL-OR-NOTHING]] [--configuration CONFIGURATION] [--em EM] [--conn CONN] [--] [<version>]
  ERR    ERR
  ERR    ERR
  ERR  In Process.php line 269:
  ERR
  ERR    [Symfony\Component\Process\Exception\ProcessFailedException]
  ERR    The command "'/usr/local/bin/php' 'vendor/doctrine/migrations/bin/doctrine-
  ERR    migrations.php' 'migrations:migrate' '--no-interaction'" failed.
  ERR
  ERR    Exit Code: 1(General error)
  ERR
  ERR    Working directory: /etc/shlink
  ERR
  ERR    Output:
  ERR    ================
  ERR
  ERR
  ERR    Error Output:
  ERR    ================
  ERR
  ERR    In InvalidDirectory.php line 15:
  ERR
  ERR
  ERR      [Doctrine\Migrations\Finder\Exception\InvalidDirectory]
  ERR
  ERR      Cannot load migrations from "data/migrations" because it is not a valid d
  ERR    ir
  ERR      ectory
  ERR
  ERR
  ERR
  ERR
  ERR    Exception trace:
  ERR      at /etc/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Finder/
  ERR    Exception/InvalidDirectory.php:15
  ERR     Doctrine\Migrations\Finder\Exception\InvalidDirectory::new() at /etc/shlin
  ERR    k/vendor/doctrine/migrations/lib/Doctrine/Migrations/Finder/Finder.php:35
  ERR     Doctrine\Migrations\Finder\Finder->getRealPath() at /etc/shlink/vendor/doc
  ERR    trine/migrations/lib/Doctrine/Migrations/Finder/GlobFinder.php:20
  ERR     Doctrine\Migrations\Finder\GlobFinder->findMigrations() at /etc/shlink/ven
  ERR    dor/doctrine/migrations/lib/Doctrine/Migrations/FilesystemMigrationsReposit
  ERR    ory.php:132
  ERR     Doctrine\Migrations\FilesystemMigrationsRepository->loadMigrationsFromDire
  ERR    ctories() at /etc/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations
  ERR    /FilesystemMigrationsRepository.php:108
  ERR     Doctrine\Migrations\FilesystemMigrationsRepository->getMigrations() at /et
  ERR    c/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Tools/Console/C
  ERR    ommand/MigrateCommand.php:160
  ERR     Doctrine\Migrations\Tools\Console\Command\MigrateCommand->execute() at /et
  ERR    c/shlink/vendor/symfony/console/Command/Command.php:326
  ERR     Symfony\Component\Console\Command\Command->run() at /etc/shlink/vendor/sym
  ERR    fony/console/Application.php:1078
  ERR     Symfony\Component\Console\Application->doRunCommand() at /etc/shlink/vendo
  ERR    r/symfony/console/Application.php:324
  ERR     Symfony\Component\Console\Application->doRun() at /etc/shlink/vendor/symfo
  ERR    ny/console/Application.php:175
  ERR     Symfony\Component\Console\Application->run() at /etc/shlink/vendor/doctrin
  ERR    e/migrations/lib/Doctrine/Migrations/Tools/Console/ConsoleRunner.php:95
  ERR     Doctrine\Migrations\Tools\Console\ConsoleRunner::run() at /etc/shlink/vend
  ERR    or/doctrine/migrations/bin/doctrine-migrations.php:45
  ERR     Doctrine\Migrations\{closure}() at /etc/shlink/vendor/doctrine/migrations/
  ERR    bin/doctrine-migrations.php:46
  ERR
  ERR    migrations:migrate [--write-sql [WRITE-SQL]] [--dry-run] [--query-time] [--
  ERR    allow-no-migration] [--all-or-nothing [ALL-OR-NOTHING]] [--configuration CO
  ERR    NFIGURATION] [--em EM] [--conn CONN] [--] [<version>]
  ERR
  ERR
  ERR  Exception trace:
  ERR    at /etc/shlink/vendor/symfony/process/Process.php:269
  ERR   Symfony\Component\Process\Process->mustRun() at /etc/shlink/module/CLI/src/Util/ProcessRunner.php:48
  ERR   Shlinkio\Shlink\CLI\Util\ProcessRunner->run() at /etc/shlink/module/CLI/src/Command/Db/AbstractDatabaseCommand.php:30
  ERR   Shlinkio\Shlink\CLI\Command\Db\AbstractDatabaseCommand->runPhpCommand() at /etc/shlink/module/CLI/src/Command/Db/MigrateDatabaseCommand.php:31
  ERR   Shlinkio\Shlink\CLI\Command\Db\MigrateDatabaseCommand->lockedExecute() at /etc/shlink/module/CLI/src/Command/Util/AbstractLockedCommand.php:35
  ERR   Shlinkio\Shlink\CLI\Command\Util\AbstractLockedCommand->execute() at /etc/shlink/vendor/symfony/console/Command/Command.php:326
  ERR   Symfony\Component\Console\Command\Command->run() at /etc/shlink/vendor/symfony/console/Application.php:1078
  ERR   Symfony\Component\Console\Application->doRunCommand() at /etc/shlink/vendor/symfony/console/Application.php:324
  ERR   Symfony\Component\Console\Application->doRun() at /etc/shlink/vendor/symfony/console/Application.php:175
  ERR   Symfony\Component\Console\Application->run() at /etc/shlink/bin/cli:10
  ERR
  ERR  db:migrate
  ERR
  ERR
  RES  1 Command did not run successfully
Generating proxies... [Running "/usr/local/bin/php bin/doctrine orm:generate-proxies"]   RUN  '/usr/local/bin/php' 'bin/doctrine' 'orm:generate-proxies'
  ERR   Processing entity "Shlinkio\Shlink\Core\Visit\Entity\VisitLocation"
  ERR   Processing entity "Shlinkio\Shlink\Core\ShortUrl\Entity\DeviceLongUrl"
  ERR   Processing entity "Shlinkio\Shlink\Core\ShortUrl\Entity\ShortUrl"
  ERR   Processing entity "Shlinkio\Shlink\Core\Tag\Entity\Tag"
  ERR   Processing entity "Shlinkio\Shlink\Core\Domain\Entity\Domain"
  ERR   Processing entity "Shlinkio\Shlink\Core\Visit\Entity\Visit"
  ERR   Processing entity "Shlinkio\Shlink\Rest\Entity\ApiKeyRole"
  ERR   Processing entity "Shlinkio\Shlink\Rest\Entity\ApiKey"
  ERR
  ERR   Proxy classes generated to "/etc/shlink/data/proxies"
  ERR
  RES  Command ran successfully
 Success!
Clearing entities cache... [Running "/usr/local/bin/php bin/doctrine orm:clear-cache:metadata"]   RUN  '/usr/local/bin/php' 'bin/doctrine' 'orm:clear-cache:metadata'
  ERR
  ERR   // Clearing all Metadata cache entries
  ERR
  ERR   [OK] Successfully deleted cache entries.
  ERR
  ERR
  RES  Command ran successfully
 Success!
Downloading GeoLite2 db file... [Running "/usr/local/bin/php bin/cli visit:download-db"]   RUN  '/usr/local/bin/php' 'bin/cli' 'visit:download-db'
  OUT
  OUT   [INFO] GeoLite2 db file is up to date.
  OUT
  OUT
  RES  Command ran successfully
 Success!

It looks like there are more directories in play here, maybe you just need a single place for all these temp file operations to use?

acelaya commented 10 months ago

The only directory you need to mount is /etc/shlink/data, but you need to make sure its contents are not replaced with an empty directory, because database migrations are located there as well.

I'm not sure if that's possible though. In retrospect, I think it probably would have made sense that migrations were located somewhere else, but I think the migrations tool tracks their path, so I'm not sure if that can be changed now. I'll check.

acelaya commented 10 months ago

Hmmm. Looks like some proxies are written to tmp first and then moved to Shlink's folder.

I'll check if this can be configured.

Writing to /tmp seems to be an implementation detail of a dependency down the line, from the DI container used by Shlink, which can generate those proxies. For some reason they write the file to /tmp and then try to move it to the configured destination.

Looking at the error, it seems writing to /tmp worked, and only moving the file to /etc/shlink/data failed, so I guess mounting the volume should cover this.

onedr0p commented 10 months ago

I tried mounting /etc/shlink/data to a emptyDir but had the error mentioned here, as you mentioned it looks like it is clobbering that directory of all existing files.

acelaya commented 10 months ago

I tried mounting /etc/shlink/data to a emptyDir but had the error mentioned here, as you mentioned it looks like it is clobbering that directory of all existing files.

I have already verified it is possible to move migrations somewhere else, so that everything inside the data dir is stuff generated at runtime.

I also need to check if it can be shipped as an empty dir, ensuring subdirs are created as required. At the moment, a few of those subdirs need to exist for everything to work, so mounting an empty dir would still fail even without taking migrations into consideration.

acelaya commented 10 months ago

I'm pretty confident that, as a workaround, you should be good to go by mounting empty dirs to /etc/shlink/data/proxies and /etc/shlink/data/locks.

If it still fails, you may need to mount /tmp as well.

In the meantime I'll continue investigating if it's possible to ship Shlink with an empty /etc/shlink/data.

acelaya commented 10 months ago

I'm pretty confident that, as a workaround, you should be good to go by mounting empty dirs to /etc/shlink/data/proxies and /etc/shlink/data/locks.

Nah, scratch that. The GeoLite db downloading needs to write directly in /etc/shlink/data 🤦🏼

onedr0p commented 10 months ago

Looks like there is also a problem here... The cache directory "data/cache" is not writable in /etc/shlink/vendor/mezzio/mezzio-fastroute/src/FastRouteRouter.php:558

acelaya commented 10 months ago

Looks like there is also a problem here... The cache directory "data/cache" is not writable in /etc/shlink/vendor/mezzio/mezzio-fastroute/src/FastRouteRouter.php:558

This is not needed when running Shlink with RoadRunner or openswoole. I'll change it as part of the other modifications.

acelaya commented 10 months ago

This is what I have finally decided to do.

  1. Move migrations somewhere else. They are in fact source code, so the location was not correct anyway.
  2. Update the docker entrypoint script to create the required data subdirs, only if they don't exist. This will allow to mount an empty dir at /etc/shlink/data.
  3. Keep everything else inside data dir as is. That way, this change won't affect others, specially those not using the docker image.

With this, the only consideration is the fact that one of the deps needs to write in /tmp, but I don't think that can be avoided.

Worst case scenario, you will have to mount /tmp as well, but according to the error above, my suspicion is that this is working already.

acelaya commented 10 months ago

I have just released v3.7.3-beta.1, which includes the changes explained above.

The 3.7.3-beta.1-non-root docker image has been already published, if you want to try it.

onedr0p commented 10 months ago

Confirmed working with setting an emptyDir to /tmp and /etc/shlink/data 🎉