microsoft / linux-package-repositories

Microsoft Packaged Linux Software (DEBs, RPMs, etc) are hosted on packages.microsoft.com (PMC) made available as native Linux repositories for use with package managers like APT, YUM, etc.
https://packages.microsoft.com
MIT License
71 stars 17 forks source link

unixodbc packages conflict with upstream packages on Ubuntu 22.04 and Debian 11 #39

Closed dlech closed 1 year ago

dlech commented 1 year ago

Describe the issue I'm getting errors from apt about unixodbc packages.

$ sudo apt install -f
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Correcting dependencies... Done
The following additional packages will be installed:
  libodbc1 odbcinst odbcinst1debian2
Suggested packages:
  msodbcsql17 unixodbc-bin
The following NEW packages will be installed:
  libodbc1 odbcinst odbcinst1debian2
0 upgraded, 3 newly installed, 0 to remove and 9 not upgraded.
2 not fully installed or removed.
Need to get 0 B/607 kB of archives.
After this operation, 945 kB of additional disk space will be used.
Do you want to continue? [Y/n] 
(Reading database ... 754326 files and directories currently installed.)
Preparing to unpack .../odbcinst_2.3.11_amd64.deb ...
Unpacking odbcinst (2.3.11) ...
dpkg: error processing archive /var/cache/apt/archives/odbcinst_2.3.11_amd64.deb
 (--unpack):
 trying to overwrite '/etc/odbc.ini', which is also in package unixodbc-common 2
.3.9-5
Preparing to unpack .../odbcinst1debian2_2.3.11_amd64.deb ...
Unpacking odbcinst1debian2:amd64 (2.3.11) ...
dpkg: error processing archive /var/cache/apt/archives/odbcinst1debian2_2.3.11_a
md64.deb (--unpack):
 trying to overwrite '/usr/lib/x86_64-linux-gnu/libodbcinst.so.2.0.0', which is 
also in package libodbcinst2:amd64 2.3.9-5
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Preparing to unpack .../libodbc1_2.3.11_amd64.deb ...
Unpacking libodbc1:amd64 (2.3.11) ...
dpkg: error processing archive /var/cache/apt/archives/libodbc1_2.3.11_amd64.deb
 (--unpack):
 trying to overwrite '/usr/lib/x86_64-linux-gnu/libodbc.so.2.0.0', which is also
 in package libodbc2:amd64 2.3.9-5
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/cache/apt/archives/odbcinst_2.3.11_amd64.deb
 /var/cache/apt/archives/odbcinst1debian2_2.3.11_amd64.deb
 /var/cache/apt/archives/libodbc1_2.3.11_amd64.deb

When did the issue occur? 14 Febuary, 2023

If applicable, what package did you attempt to install, and from which repo?

Just doing automatic apt update. libodbc2 is used by many packages (unrelated to Microsoft). I have http://packages.microsoft.com/ in my apt sources for VS Code.

Steps to Reproduce

Actual Result

Apt fails because of conflicts. The Microsoft version of the package is shipping the v2 library in a libodbc1 package instead of a libodbc2 package. Apt tries to install both at the same time and it can't because both packages try to install the same file.

Expected Result

Microsoft should not be shipping updates to upstream packages in a general package archive like this. Or at minimum, it should be opt-in rather than opt out.

Screenshots

Additional context

rajpawar02 commented 1 year ago

I'm having the same problem running CI on the GitHub action runner ( Ubuntu 22.04.1). The details of the error are as follows:

Preparing to unpack .../0-libodbc1_2.3.11_amd64.deb ...
Unpacking libodbc1:amd64 (2.3.11) ...
dpkg: error processing archive /tmp/apt-dpkg-install-nDLQ5L/0-libodbc1_2.3.11_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/x86_64-linux-gnu/libodbc.so.2.0.0', which is also in package libodbc2:amd64 2.3.9-5
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Selecting previously unselected package odbcinst1debian2:amd64.
Preparing to unpack .../1-odbcinst1debian2_2.3.11_amd64.deb ...
Unpacking odbcinst1debian2:amd64 (2.3.11) ...
dpkg: error processing archive /tmp/apt-dpkg-install-nDLQ5L/1-odbcinst1debian2_2.3.11_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/x86_64-linux-gnu/libodbcinst.so.2.0.0', which is also in package libodbcinst2:amd64 2.3.9-5
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Preparing to unpack .../2-odbcinst_2.3.11_amd64.deb ...
Unpacking odbcinst (2.3.11) over (2.3.9-5) ...
dpkg: error processing archive /tmp/apt-dpkg-install-nDLQ5L/2-odbcinst_2.3.11_amd64.deb (--unpack):
 trying to overwrite '/etc/odbc.ini', which is also in package unixodbc-common 2.3.9-5
Selecting previously unselected package postgresql-client.
Preparing to unpack .../3-postgresql-client_14+238_all.deb ...
Unpacking postgresql-client (14+238) ...
Preparing to unpack .../4-unixodbc_2.3.11_amd64.deb ...
Unpacking unixodbc (2.3.11) over (2.3.9-5) ...
Preparing to unpack .../5-unixodbc-dev_2.3.11_amd64.deb ...
Unpacking unixodbc-dev (2.3.11) over (2.3.9-5) ...
Errors were encountered while processing:
 /tmp/apt-dpkg-install-nDLQ5L/0-libodbc1_2.3.11_amd64.deb
 /tmp/apt-dpkg-install-nDLQ5L/1-odbcinst1debian2_2.3.11_amd64.deb
 /tmp/apt-dpkg-install-nDLQ5L/2-odbcinst_2.3.11_amd64.deb
needrestart is being skipped since dpkg has failed
E: Sub-process /usr/bin/dpkg returned an error code (1)
Error: Process completed with exit code [100](https://github.com/patterninc/shipmentczar/actions/runs/4179207866/jobs/7242081737#step:6:101).

Anyone know of a solution?

Mause commented 1 year ago

Looks like this has been raised in https://bugs.launchpad.net/ubuntu/+source/unixodbc/+bug/2006998 and https://bugs.launchpad.net/ubuntu/+source/unixodbc/+bug/2007288 (the second one includes a hack fix)

fdellwing commented 1 year ago

Having the same problem here today with with libodbc1 package.

❯ apt-cache policy libodbc1
libodbc1:
 Installiert:           2.3.9-5
 Installationskandidat: 2.3.11
 Versionstabelle:
    2.3.11 500
       500 https://packages.microsoft.com/ubuntu/22.04/prod jammy/main amd64 Packages
*** 2.3.9-5 500
       500 http://de.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
       100 /var/lib/dpkg/status
    2.3.7 500
       500 https://packages.microsoft.com/ubuntu/22.04/prod jammy/main amd64 Packages

❯ apt-cache policy libodbc2
libodbc2:
 Installiert:           2.3.9-5
 Installationskandidat: 2.3.9-5
 Versionstabelle:
*** 2.3.9-5 500
       500 http://de.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
       100 /var/lib/dpkg/status

MS seems to ship a newer libodbc1 package that tries to overwrite a .so provided by libodbc2.

ThorHexSec commented 1 year ago

Hi everyone, this problem persist in Ubuntu alternatives Distro (Pop OS, Zorin and Linux Mint).

This problem broken command apt dist-upgrade and Erlang Kerl Compile (Unixodbc is necessary to Erlang Kerl native or ASDF).

This temporary solution for me:

sudo dpkg --remove unixodbc

sudo dpkg --remove unixodbc-dev

I Have install erlang with snap and vscode is snap temporally until the correction.

I'm commenting on this issue to keep track of possible solutions.

fdellwing commented 1 year ago

The only feasible long term solution is for MS to fix their repo.

manitgupta commented 1 year ago

Facing the same issue with out of the blue in our GIthub action runner today:

(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 236141 files and directories currently installed.)
Preparing to unpack .../0-libodbc1_2.3.11_amd64.deb ...
Unpacking libodbc1:amd64 (2.3.11) ...
dpkg: error processing archive /tmp/apt-dpkg-install-6gccu1/0-libodbc1_2.3.11_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/x86_64-linux-gnu/libodbc.so.2.0.0', which is also in package libodbc2:amd64 2.3.9-5
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Selecting previously unselected package odbcinst1debian2:amd64.
Preparing to unpack .../1-odbcinst1debian2_2.3.11_amd64.deb ...
Unpacking odbcinst1debian2:amd64 (2.3.11) ...
dpkg: error processing archive /tmp/apt-dpkg-install-6gccu1/1-odbcinst1debian2_2.3.11_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/x86_64-linux-gnu/libodbcinst.so.2.0.0', which is also in package libodbcinst2:amd64 2.3.9-5
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Preparing to unpack .../2-odbcinst_2.3.11_amd64.deb ...
Unpacking odbcinst (2.3.11) over (2.3.9-5) ...
dpkg: error processing archive /tmp/apt-dpkg-install-6gccu1/2-odbcinst_2.3.11_amd64.deb (--unpack):
 trying to overwrite '/etc/odbc.ini', which is also in package unixodbc-common 2.3.9-5
Preparing to unpack .../3-unixodbc_2.3.11_amd64.deb ...
Unpacking unixodbc (2.3.11) over (2.3.9-5) ...
signedav commented 1 year ago

@rajpawar02 Fixed my CIs by installing the odbc stuff before adding the MS repositories according to this input https://bugs.launchpad.net/ubuntu/+source/unixodbc/+bug/2007288/comments/3

See https://github.com/opengisch/QgisModelBakerLibrary/pull/54/commits/a6dfc8eea64867290b335c7375e436b00be224fd

manitgupta commented 1 year ago

I tried the suggested workaround in the linked bug but it did not work for me -

Commit - https://github.com/manitgupta/harbourbridge/commit/e30062a9a74bc703bb58816f5e2457a80d7547a4 Failed run - https://github.com/manitgupta/harbourbridge/actions/runs/4183240613/jobs/7247347511

dlech commented 1 year ago

EDIT: for a more complete example, jump ahead to https://github.com/microsoft/linux-package-repositories/issues/39#issuecomment-1446041805

For desktop users that don't want/need the Microsoft version of unixodbc, you can add the following to /etc/apt/preferences.d/microsoft:

Package: unixodbc
Pin: origin "packages.microsoft.com"
Pin-Priority: 100
Package: unixodbc-dev
Pin: origin "packages.microsoft.com"
Pin-Priority: 100
fdellwing commented 1 year ago

Even though no one from MS wrote something here, the package seems to habe been removed from the repository again and therefore this issue should no longer exist.

fjudith commented 1 year ago

Even though no one from MS wrote something here, the package seems to habe been removed from the repository again and therefore this issue should no longer exist.

I confirm, the issue seems to be solved. No more install erreor.

ThorHexSec commented 1 year ago

I execute dist-upgrade in my Ubuntu 22.04 is not having issue in Microsoft repository.

Installation proceed normaly in Unixodbc and Unixodbc-dev.

manitgupta commented 1 year ago

Yes, solved now.

falkben commented 1 year ago

running this again this morning, I'm still seeing 2.3.11 being installed with a reference to the wrong header

FROM --platform=linux/amd64 python:3.8-bullseye

RUN \
    curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
    && curl https://packages.microsoft.com/config/debian/11/prod.list > /etc/apt/sources.list.d/mssql-release.list \
    && export DEBIAN_FRONTEND=noninteractive \
    && apt-get update \
    && ACCEPT_EULA=Y apt-get install --yes \
    msodbcsql17 \
    unixodbc-dev \
    && apt-get clean && rm -rf /var/lib/apt/lists/* 

RUN \
    pip install --no-binary :all: pyodbc

pinning unixodbc-dev=2.3.7 unixodbc=2.3.7 odbcinst1debian2=2.3.7 odbcinst=2.3.7 still works

fdellwing commented 1 year ago

@falkben This was so far about the Ubuntu 22 repo. It seems that the Debian 11 repo is also and still affected (while Ubuntu is fixed).

ethrya commented 1 year ago

I understand that I am still having this issue on Ubuntu 20.04.

It looks like the packages have not been removed from the packages files as I can still see them (e.g. libodbc) here: https://packages.microsoft.com/ubuntu/20.04/prod/dists/focal/main/)

alanplarium commented 1 year ago

It is still failing for me. Anyone else still could not resolve the issue?

David-Engel commented 1 year ago

This should be addressed now. The unixodbc 2.3.11 packages were removed for Ubuntu 22.04 and higher. Use the unixodbc packages provided by Ubuntu. For Debian 10, 11, and Ubuntu 20.04, 18.04, an updated package, 2.3.11-1, was published with a fix for the header issue.

alanplarium commented 1 year ago

@David-Engel It is still failing for me. I am building a docker image and fails in one of these steps:

RUN sudo su \ && curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \ && curl https://packages.microsoft.com/config/ubuntu/20.04/prod.list

I get these errors: dpkg: error processing archive /var/cache/apt/archives/libodbc1_2.3.11-1_amd64.deb (--unpack): dpkg: error processing archive /var/cache/apt/archives/odbcinst1debian2_2.3.11-1_amd64.deb (--unpack):

Anything you think I should change?

sheikx1221 commented 1 year ago

removing these packages is a pain you don't want to feel here is the therapy

sudo dpkg --remove unixodbc-dev unixodbc

then you need to edit postrm script of msodbcsql17 using root sudo vim /var/lib/dpkg/info/msodbcsql17.postrm

comment the two lines in postrm that start with odbcinst and db_purge in the remove section and save

sudo apt-get remove msodbcsql17 sudo apt remove unixodbc unixodbc-dev odbcinst odbcinst1debian2 libodbc1 msodbcsql17 mssql-tools

Then follow this tutorial for 22.04 ubuntu desktop

geekdenz commented 1 year ago

I worked around and solved it by

sudo vi /etc/apt/sources.list.d/msprod.list

Comment out the line(s):

#deb [arch=amd64,armhf,arm64] https://packages.microsoft.com/ubuntu/20.04/prod focal main

Run

sudo apt update && sudo apt --fix-broken install

Then reinstall MS products by doing the reverse if desired.

Yaroslav285 commented 1 year ago

Hello, I tried to use Ubuntu's MS repo in Debian 11 and it works.

https://packages.microsoft.com/config/ubuntu/22.04/prod.list

For Debian's repo, it generates an error


#4 46.78 libtool: link: warning: library `/usr/lib/x86_64-linux-gnu/libodbc.la' was moved.
#4 46.79 grep: /usr/lib/x86_64-linux-gnu/libltdl.la: No such file or directory
#4 46.79 /bin/sed: can't read /usr/lib/x86_64-linux-gnu/libltdl.la: No such file or directory
#4 46.79 libtool: link: `/usr/lib/x86_64-linux-gnu/libltdl.la' is not a valid libtool archive
#4 46.79 make: *** [Makefile:249: sqlsrv.la] Error 1
#4 46.79 ERROR: `make' failed```

after running the command

`pecl install sqlsrv`
lwohlhart commented 1 year ago

We are running php with mssql driver in our production server cluster. ( currently all machines are on ubuntu 18.04, in the process of upgrading to 22.04 ) We are also currently solving the issue by pinning the repository such that we don't recieve the problematic unixodbc and unixodbc-dev packages hosted in the microsoft repo. Similar to https://github.com/microsoft/linux-package-repositories/issues/39#issuecomment-1431791428 pecl installation of the drivers would break as other described before me.

@David-Engel could you please comment on why it's necessary for microsoft to host these packages in general? Are we going to run into other sorts of incompatiblities if we don't install this version of the package? imho this would be a bit odd for a package like unixodbc

moerkey commented 1 year ago

For desktop users that don't want/need the Microsoft version of unixodbc, you can add the following to /etc/apt/preferences.d/microsoft:

Package: unixodbc
Pin: origin "packages.microsoft.com"
Pin-Priority: 100
Package: unixodbc-dev
Pin: origin "packages.microsoft.com"
Pin-Priority: 100

For me, that wasn´t sufficient under Ubuntu 18.04. I think it is important to add a new line between the two sections like this.

Package: unixodbc
Pin: origin "packages.microsoft.com"
Pin-Priority: 100

Package: unixodbc-dev
Pin: origin "packages.microsoft.com"
Pin-Priority: 100

Also I had to add more packages. So my total pinning looks like this.

# https://github.com/microsoft/linux-package-repositories/issues/39#issuecomment-1431791428

Package: unixodbc
Pin: origin "packages.microsoft.com"
Pin-Priority: 100

Package: unixodbc-dev
Pin: origin "packages.microsoft.com"
Pin-Priority: 100

Package: libodbc1:amd64
Pin: origin "packages.microsoft.com"
Pin-Priority: 100

Package: odbcinst
Pin: origin "packages.microsoft.com"
Pin-Priority: 100

Package: odbcinst1debian2:amd64
Pin: origin "packages.microsoft.com"
Pin-Priority: 100

Btw I didn´t understand the intention but nice way to solve it via lowering the pin priority.

lucasmichel commented 1 year ago

In case anyone has the same problem I solved it as follows:

change url https://packages.microsoft.com/config/debian/11/prod.list from https://packages.microsoft.com/config/ubuntu/22.10/prod.list

change msodbcsql17 from msodbcsql18

going like this:

RUN cat /etc/os-release \
  && apt-get update \
  && curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
  && curl https://packages.microsoft.com/config/ubuntu/22.10/prod.list \
      > /etc/apt/sources.list.d/mssql-release.list \
  && apt-get install -y --no-install-recommends apt-transport-https \
  && apt-get update \
  && ACCEPT_EULA=Y apt-get -y --no-install-recommends install unixodbc-dev msodbcsql18  
Yaroslav285 commented 1 year ago

Hello @lucasmichel, it works for me too. That's what I mentioned in my comment

Hello, I tried to use Ubuntu's MS repo in Debian 11 and it works.

https://packages.microsoft.com/config/ubuntu/22.04/prod.list

For Debian's repo, it generates an error

#4 46.78 libtool: link: warning: library `/usr/lib/x86_64-linux-gnu/libodbc.la' was moved.
#4 46.79 grep: /usr/lib/x86_64-linux-gnu/libltdl.la: No such file or directory
#4 46.79 /bin/sed: can't read /usr/lib/x86_64-linux-gnu/libltdl.la: No such file or directory
#4 46.79 libtool: link: `/usr/lib/x86_64-linux-gnu/libltdl.la' is not a valid libtool archive
#4 46.79 make: *** [Makefile:249: sqlsrv.la] Error 1
#4 46.79 ERROR: `make' failed```

after running the command

`pecl install sqlsrv`

I hope Microsoft engineers are going to perform some actions to resolve the issue as well for Debian 11.

David-Engel commented 1 year ago

could you please comment on why it's necessary for microsoft to host these packages in general? Are we going to run into other sorts of incompatiblities if we don't install this version of the package?

@lwohlhart

We provide builds of unixodbc because we submit bug fixes to unixodbc that were found in the course of troubleshooting customer issues. Linux distributions tend to lag quite a bit behind in updating to newer versions of unixodbc (some more than others). 2.3.7 should be considered a minimum version to run, bug wise, with 2.3.9 being the next best step, in our experience. You should not run into incompatibilities if you don't install the latest version.

Regards, David

tclrl commented 1 year ago

hi, same problem here. i use debian 11. i've installed theses packages : php8.2 : https://packages.sury.org/ 8.2.3-1+0~20230214.16+debian11~1.gbpd66462 libodbc1 : http://deb.debian.org/debian 2.3.6-0.1+b1 unixodbc : http://deb.debian.org/debian 2.3.6-0.1+b1 unixodbc-dev : http://deb.debian.org/debian 2.3.6-0.1+b1 msodbcsql18 : https://packages.microsoft.com/ubuntu/22.10/prod 18.2.1.1-1

it's work but the package went from ubuntu

sdherr commented 1 year ago

Is there still a problem here or can I close this issue? I've tested things out in Debian 11 and Ubuntu 18.04 and it seems to work currently. If something is still broken we need a more clear reproducer.

From an arbitrary machine with docker (or similar) installed: docker run -it debain:11 (or ubuntu:18.04):

apt-get update
apt-get install wget blender
wget https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb
# or, on Ubuntu:
# wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb
apt-get install ./packages-microsoft-prod.deb
apt-get update
apt-get upgrade
# -> works
dlech commented 1 year ago

It's been a few weeks since any chatter, so I'll go ahead and close it and see if anyone objects.

fcorrigabb commented 1 year ago

The problem is still there.

pborquez commented 1 year ago

Until today, the problem is still here but this is the way I solved the issue using a Debian image:

FROM php:8.2.1-apache-bullseye as production

RUN set -eux; \
  apt-get update; \
  apt-get install -y --no-install-recommends \
    gnupg \
    wget \
    curl \
  ; \
  curl -SL --progress-bar https://packages.microsoft.com/keys/microsoft.asc | apt-key add -; \
  curl -SL --progress-bar https://packages.microsoft.com/config/ubuntu/22.04/prod.list > /etc/apt/sources.list.d/mssql-release.list; \
  apt-get update; \
  ACCEPT_EULA=Y apt-get install -y msodbcsql18; \
  apt-get install -y unixodbc-dev; \
  sed -i -E 's/(CipherString\s*=\s*DEFAULT@SECLEVEL=)2/\11/' /etc/ssl/openssl.cnf; \
  rm -rf /var/lib/apt/lists/*;

RUN set -eux; \
  pecl install \
    sqlsrv \
    pdo_sqlsrv \
  ; \
  docker-php-ext-enable \
    sqlsrv \
    pdo_sqlsrv \
  ; \
  echo "extension= pdo_sqlsrv.so" >> `php --ini | grep "Loaded Configuration" | sed -e "s|.*:\s*||"`; \
  echo "extension= sqlsrv.so" >> `php --ini | grep "Loaded Configuration" | sed -e "s|.*:\s*||"`;
rb12345 commented 1 year ago

It might be worth noting that this breaks upgrades from bullseye to bookworm if anything else depends on the library.

ameryu716 commented 1 year ago

> @pborquez curl -SL --progress-bar https://packages.microsoft.com/config/ubuntu/22.04/prod.list > /etc/apt/sources.list.d/mssql-release.list; \ Acturally thanks!! I was freed from the bondage of two hours.

killerkalamari commented 1 year ago

This got me past the problem on a Debian 11 to 12 (bullseye to bookworm) dist-upgrade: sudo apt remove python3-gdal libgdal32 libgdal28 libodbc1. Nothing else seemed to depend on those in my case.

stlrnz commented 1 year ago

I can confirm that I still have to use the ubuntu repository for my debian 11 docker container to install the PHP sqlsrv extension.

RUN apt install -y gpg \
    && curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
    && curl https://packages.microsoft.com/config/ubuntu/22.04/prod.list > /etc/apt/sources.list.d/mssql-release.list \
    && apt update

RUN ACCEPT_EULA=Y apt install -y msodbcsql18 mssql-tools18 \
    && apt install -y unixodbc-dev \
    && pecl install sqlsrv \
    && docker-php-ext-enable sqlsrv

If I switch to https://packages.microsoft.com/config/debian/11/prod.list the build will fail.

mbomb007 commented 1 year ago

Yes, this still fails when using debian. I have to include the following line in my script:

apt-get install -y --allow-downgrades unixodbc-dev=2.3.7 unixodbc=2.3.7 odbcinst1debian2=2.3.7 odbcinst=2.3.7
mangelozzi commented 1 year ago

Why is this issue closed? It is still broken with Ubuntu 22.04.

fcorrigabb commented 1 year ago

Because devs do not care. This is a widespread issue and it's causing a pletora of problems... but devs do not care to fix the conflicts.

mbomb007 commented 1 year ago

Is there still a problem here or can I close this issue? I've tested things out in Debian 11 and Ubuntu 18.04 and it seems to work currently. If something is still broken we need a more clear reproducer.

From an arbitrary machine with docker (or similar) installed: docker run -it debain:11 (or ubuntu:18.04):

apt-get update
apt-get install wget blender
wget https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb
# or, on Ubuntu:
# wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb
apt-get install ./packages-microsoft-prod.deb
apt-get update
apt-get upgrade
# -> works

They did say they need a clear way to reproduce the issue. Could someone create that?

fcorrigabb commented 1 year ago

It's in the initial report, it works only when unixodbc ver 2.3.6 is already installed and, as you know, 2.3.6 is outdated and buggy.

sdherr commented 1 year ago

Okay, I can now reproduce the issue. Building this Dockerfile for example fails:

FROM debian:11

RUN apt update
RUN apt install -y gpg curl php-dev
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/debian/11/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt update

RUN ACCEPT_EULA=Y apt install -y msodbcsql18 mssql-tools18
RUN apt install -y unixodbc-dev
RUN pecl install sqlsrv

This appears to be a duplicate of this issue, where they note not only the "use the Ubuntu repo" workaround but also a "use an older version of unixodbc" workaround:

&& ACCEPT_EULA=Y apt-get install -y msodbcsql17 odbcinst=2.3.7 odbcinst1debian2=2.3.7 unixodbc-dev=2.3.7 unixodbc=2.3.7 \

The root cause appears to be (and here I'm only reading the comments in that issue) that these packages were built from a broken version of upstream unixodbc=2.3.11, which was later fixed in unixodbc=2.3.11-2. Fixing the packages in the Microsoft repo would involve rebuilding them on the fixed -2 version.

I recommend everyone move this discussion to that issue, since it's actually filed against the team that can help here. I'll also ping them internally with the reproducer and explanation.

SirArthur7 commented 1 year ago

I found the following code snippet from Microsoft page to Install the Microsoft ODBC driver for SQL Server (Linux), and it works perfectly fine-

curl https://packages.microsoft.com/keys/microsoft.asc | sudo tee /etc/apt/trusted.gpg.d/microsoft.asc
curl https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/prod.list | sudo tee /etc/apt/sources.list.d/mssql-release.list
sudo apt-get update
sudo ACCEPT_EULA=Y apt-get install -y msodbcsql17
mbomb007 commented 1 year ago

I found the following code snippet from Microsoft page to Install the Microsoft ODBC driver for SQL Server (Linux), and it works perfectly fine-

curl https://packages.microsoft.com/keys/microsoft.asc | sudo tee /etc/apt/trusted.gpg.d/microsoft.asc
curl https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/prod.list | sudo tee /etc/apt/sources.list.d/mssql-release.list
sudo apt-get update
sudo ACCEPT_EULA=Y apt-get install -y msodbcsql17

You should know that adding a key to /etc/apt/trusted.gpg.d is insecure:

Adding a key to /etc/apt/trusted.gpg.d is insecure because it adds the key for all repositories. This is exactly why apt-key had to be deprecated.

SirArthur7 commented 1 year ago

You should know that adding a key to /etc/apt/trusted.gpg.d is insecure:

Adding a key to /etc/apt/trusted.gpg.d is insecure because it adds the key for all repositories. This is exactly why apt-key had to be deprecated.

Hey @mbomb007 thanks for letting me know, I wasn't aware of that. So what do you suggest I should do in order to get around this issue?

Actually I found another solution that works perfectly fine too, turned out I just needed to install unixodbc-dev before the rest-

apt-get install -y unixodbc-dev
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
curl https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/prod.list > /etc/apt/sources.list.d/mssql-release.list
sudo apt-get update
sudo ACCEPT_EULA=Y apt-get -q -y install msodbcsql17
mbearup commented 1 year ago

@SirArthur7 the recommended path is to use the signed-by option as explained in the Debian docs. This option specifies where the pubkey for a given repo can be found. We (packages.microsoft.com) are updating our configs to align with this standard (see the Debian 12 configs as an example). I can't speak to apt-key but I know that add-apt-repository still installs pubkeys to the "wrong" location (/etc/apt/trusted.gpg.d).

mbomb007 commented 1 year ago

The new file format for apt sources is known as DEB822. As an example, for MariaDB it looks like:

Types: deb
URIs: https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/ubuntu
Suites: jammy
Components: main main/debug
Architectures: amd64 arm64
Signed-By: /etc/apt/keyrings/mariadb-keyring-2019.gpg

[... more entries ...]

And the keyring was obtained with

sudo curl -LsSo /etc/apt/keyrings/mariadb-keyring-2019.gpg https://supplychain.mariadb.com/mariadb-keyring-2019.gpg

I manually converted the default-style file that was created when using the setup script into the new DEB822 format.