timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
17.67k stars 883 forks source link

[Bug]: "trying to overwrite" errors when switching to/from OSS/TSL version #5963

Open perlun opened 1 year ago

perlun commented 1 year ago

What type of bug is this?

Configuration

What subsystems and features are affected?

Packaging

What happened?

When the following packages are installed:

$ dpkg -l | grep timescaledb
ii  timescaledb-2-loader-postgresql-13     2.9.0~ubuntu18.04                               amd64        The loader for TimescaleDB to load individual versions.
ii  timescaledb-2-oss-postgresql-13        2.9.0~ubuntu18.04                               amd64        An open-source time-series database based on PostgreSQL, as an extension.
ii  timescaledb-toolkit-postgresql-13      1:1.13.0~ubuntu18.04                            amd64        Library of analytical hyperfunctions, time-series pipelining, and other SQL utilities, compatible with TimescaleDB and PostgreSQL
ii  timescaledb-tools                      0.14.3~ubuntu18.04                              amd64        A suite of tools that can be used with TimescaleDB.

..and the user tries to switch to the TSL-licensed (Timescale Community Edition) version of the extension, the following error appears:

$ sudo apt-get install timescaledb-2-2.9.0-postgresql-13 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  timescaledb-2-2.9.0-postgresql-13
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/2 997 kB of archives.
After this operation, 20,1 MB of additional disk space will be used.
(Reading database ... 152554 files and directories currently installed.)
Preparing to unpack .../timescaledb-2-2.9.0-postgresql-13_2.9.0~ubuntu18.04_amd64.deb ...
Unpacking timescaledb-2-2.9.0-postgresql-13 (2.9.0~ubuntu18.04) ...
dpkg: error processing archive /var/cache/apt/archives/timescaledb-2-2.9.0-postgresql-13_2.9.0~ubuntu18.04_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/postgresql/13/lib/timescaledb-2.9.0.so', which is also in package timescaledb-2-oss-postgresql-13 2.9.0~ubuntu18.04
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/cache/apt/archives/timescaledb-2-2.9.0-postgresql-13_2.9.0~ubuntu18.04_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Suggested fix

I believe the problem here is that the package(s) are missing proper Conflicts settings (https://www.debian.org/doc/debian-policy/ch-relationships.html#conflicting-binary-packages-conflicts). The timescaledb-2-2.9.0-postgresql-13 should Conflicts: timescaledb-2-oss-postgresql-13 and vice versa (note that since the extension is published with multiple package names, the Conflicts should probably be set up for all the relevant conflicting packages to make it as fool-proof as possible).

Note that despite the fact that this issue mentions Ubuntu 18.04, I'm quite certain that the same problem exists on the Ubuntu 20.04 and 22.04 packages as well. (I.e., please don't dismiss this as "irrelevant because of 18.04 EOL". :grin:)

TimescaleDB version affected

2.9.0

PostgreSQL version used

13

What operating system did you use?

Ubuntu 18.04

What installation method did you use?

Deb/Apt

What platform did you run on?

On prem/Self-hosted

Relevant log output and stack trace

No response

How can we reproduce the bug?

See the steps above.
perlun commented 1 year ago

If you look into (and hopefully fix) this bug, it would be great to get this (also .deb package related) issue looked into at the same time: https://github.com/timescale/timescaledb/issues/4445#issuecomment-1674329038. :pray:

mkindahl commented 1 year ago

@perlun Thank you for the bug report! If this is blocking you right now, you might want to test this to see if it works:

sudo apt-get -o Dpkg::Options::="--force-overwrite" install timescaledb-2-2.9.0-postgresql-13 
mkindahl commented 1 year ago

@perlun Each of the packages have some overlap between what files are actually present, mostly to handle upgrades between different versions, and we are trying to use Replaces rather than Conflicts for the packages.

Going to take a closer look and see what is going on with the package and see if we can get rid of the error.

mkindahl commented 1 year ago

Trivial to reproduce using this Dockerfile:

FROM ubuntu:18.04
RUN apt-get -qy update && \
    DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -qy install gnupg postgresql-common apt-transport-https lsb-release wget
RUN echo | /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
RUN <<EOT
echo "deb https://packagecloud.io/timescale/timescaledb/ubuntu/ $(lsb_release -c -s) main" | tee /etc/apt/sources.list.d/timescaledb.list
wget --quiet -O - https://packagecloud.io/timescale/timescaledb/gpgkey | tee /etc/apt/trusted.gpg.d/timescale.asc
EOT
RUN apt-get -qy update && \
    DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -qy install timescaledb-2-oss-postgresql-13
mats@fury:~/issues/issue-5963$ docker run -ti $(docker build -q .) bash
root@2e8e560f1ea7:/# dpkg -l | grep timescaledb
ii  timescaledb-2-loader-postgresql-13 2.9.0~ubuntu18.04                   amd64        The loader for TimescaleDB to load individual versions.
ii  timescaledb-2-oss-postgresql-13    2.9.0~ubuntu18.04                   amd64        An open-source time-series database based on PostgreSQL, as an extension.
ii  timescaledb-toolkit-postgresql-13  1:1.13.0~ubuntu18.04                amd64        Library of analytical hyperfunctions, time-series pipelining, and other SQL utilities, compatible with TimescaleDB and PostgreSQL
ii  timescaledb-tools                  0.14.3~ubuntu18.04                  amd64        A suite of tools that can be used with TimescaleDB.
root@2e8e560f1ea7:/# apt install timescaledb-2-2.9.0-postgresql-13 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  timescaledb-2-2.9.0-postgresql-13
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 2997 kB of archives.
After this operation, 20.1 MB of additional disk space will be used.
Get:1 https://packagecloud.io/timescale/timescaledb/ubuntu bionic/main amd64 timescaledb-2-2.9.0-postgresql-13 amd64 2.9.0~ubuntu18.04 [2997 kB]
Fetched 2997 kB in 1s (2575 kB/s)                          
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package timescaledb-2-2.9.0-postgresql-13.
(Reading database ... 10396 files and directories currently installed.)
Preparing to unpack .../timescaledb-2-2.9.0-postgresql-13_2.9.0~ubuntu18.04_amd64.deb ...
Unpacking timescaledb-2-2.9.0-postgresql-13 (2.9.0~ubuntu18.04) ...
dpkg: error processing archive /var/cache/apt/archives/timescaledb-2-2.9.0-postgresql-13_2.9.0~ubuntu18.04_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/postgresql/13/lib/timescaledb-2.9.0.so', which is also in package timescaledb-2-oss-postgresql-13 2.9.0~ubuntu18.04
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/cache/apt/archives/timescaledb-2-2.9.0-postgresql-13_2.9.0~ubuntu18.04_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
perlun commented 1 year ago

@mkindahl Thanks for a fast reply, appreciated. :+1: This is not blocking me in any way, I just ran into it on a machine where we had first installed one of the packages and then wanted to switch to the other. I don't have a particularly strong opinion regarding the Conflicts vs Replaces keywords, but it does seem like a bit of an edge case to have both packages installed at the same time, right? So potentially the "Replacing whole packages" section in the linked-to Debian policy would apply, i.e. specifying both Replaces and Conflicts:

Provides: mail-transport-agent
Conflicts: mail-transport-agent
Replaces: mail-transport-agent

Either way, I think I'm quite happy with whichever solution is chosen as long as it:

  1. provides a great UX to the end-user (i.e. no errors when switching from one package to the other) and
  2. avoids the confusion of leaving both packages being installed simultaneously (note that this is me presuming that they are in essence "technically incompatible" to have installed at the same time)