Version 4.x not updating with apt update Ubuntu 20.04

[TCROC]

On Ubuntu 20.04, Podman is stuck at version 3.4.2. Running apt update doesn't update it to the 4.x version. I followed the instructions here: https://podman.io/getting-started/installation.html#linuxmint-20x.

[afbjorklund]

Those packages (from Kubic) are not updated anymore, the instructions should be removed (for Mint, like they were for Ubuntu)

https://github.com/containers/podman.io/commit/4c7bd52a56cc9bef5087da25889424ae0ee579ad

[mythi]

Those packages (from Kubic) are not updated anymore, the instructions should be removed (for Mint, like they were for Ubuntu)

has there been any announcements about kubic? we've installed buildah from it and that disappeared completely.

[TCROC]

Those packages (from Kubic) are not updated anymore, the instructions should be removed (for Mint, like they were for Ubuntu)

What is the new way to install Podman on Ubuntu 20.04?

[afbjorklund]

Here was the update: https://podman.io/blogs/2022/04/05/ubuntu-2204-lts-kubic.html

[TCROC]

Ok I see. So Podman 4.x is available on Ubuntu 22.04 if I upgrade?

[afbjorklund]

It is available on Fedora and RHEL, not on Ubuntu (yet)

https://packages.ubuntu.com/search?keywords=podman&searchon=names&exact=1&suite=all&section=all

[TCROC]

Any plans for when it will land on Ubuntu? 👀 :)

[afbjorklund]

https://packages.debian.org/search?keywords=podman&searchon=names&exact=1&suite=all&section=all

Not in "Sid" yet, see https://wiki.ubuntu.com/Upstream

[TCROC]

Ah ok cool. And am I able to add that as a PPA on Ubuntu?

[afbjorklund]

I believe you would need to build it from the source code.

• https://github.com/containers/podman/issues/13407#issuecomment-1058137350

The same goes for all the required dependencies, as well...

• https://github.com/kubernetes/minikube/issues/14198

---

More discussion:

• https://github.com/containers/podman/issues/14065
• https://github.com/containers/podman/discussions/14260
• https://github.com/containers/podman/discussions/12265

[TCROC]

What is the reasoning for not including 4.x in an Ubuntu ppa?

[lsm5]

@TCROC Ubuntu ppas are a huge PITA (at least for me (Fedora guy)). Also, no debian support from what I can tell.

RE: debian experimental repo that @afbjorklund mentioned above, you should be able to install the deb package on ubuntu, at least that's what the debian maintainer told us. I don't know if you'd be able to add it as a repo source to Ubuntu though.

Also, I have a WIP kubic repo at https://build.opensuse.org/project/show/devel:kubic:libcontainers:unstable . This will likely not be officially documented for quite some time as I cannot promise long term maintainability / support yet. But if you're willing to try it out, be my guest. This repo currently allows you to use podman v4.1.0 on debian sid and ubuntu 22.04 LTS along with the latest network stack (netavark and aardvark-dns).

Closing..

[wuestkamp]

Is there still a way to install Podman (any version) on Ubuntu 20.04? Because this throws gpg: no valid OpenPGP data found:

. /etc/os-release
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/testing/xUbuntu_${VERSION_ID}/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:testing.list
curl -L "https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/testing/xUbuntu_${VERSION_ID}/Release.key" | sudo apt-key add -
sudo apt-get update -qq
sudo apt-get -qq -y install podman

[TCROC]

@wuestkamp I suggested something over here: https://github.com/containers/podman/issues/14332

Sounds like Snap or Flatpak might be a possibility 👀. Still waiting on an official comment.

[afbjorklund]

Is there still a way to install Podman (any version) on Ubuntu 20.04?

The kubic packages are still the "best", but they do have security issues.

EDIT: Nope

[lsm5]

Given all the love for podman and ubuntu 20.04, I have restored podman on the devel:kubic:libcontainers:stable project. Again, this is the same v3.4.2 build that used to exist prior and is vulnerable to the CVEs mentioned above. So, use at your own risk.

Builds should be published in a few mins.

[crfrolik]

I'm confused by this. Why is it still so difficult to get 4.X installed on Ubuntu?

[vrothberg]

I'm confused by this. Why is it still so difficult to get 4.X installed on Ubuntu?

It's an immense amount of work. It is not "only" packaging podman but many of its runtime dependencies including config files, man pages etc. Plus maintaining all of that.

Since podman is now in the main repositories of Ubuntu, we prefer users to use these packages.

[alexanderadam]

Since podman is now in the main repositories of Ubuntu, we prefer users to use these packages.

The current Ubuntu 22.04 LTS will be supported for 10 years. So you suggest that its users are stuck with Podman 3 for ten years?

I also asked on StackOverflow for a solution but so far no one got an answer how to get Podman 4 in a sane manner that doesn't involve setting up and maintaining binaries manually.

It's probably pointless to mention again that an official PPA would solve that for everybody. :wink:

…but I also had some issues with Kernel built-in overlayfs so there's that. :shrug:

[vrothberg]

If people in the community step up to do this work, they are more than welcome.

The Podman maintainers simply don't have the capacity to ship/package new versions to all distributions since the support matrix is just too big.

…but I also had some issues with Kernel built-in overlayfs so there's that.

Please open issues on GitHub. Twitter is not a good platform to report bugs or issues; it's very easy that Tweets get lost in the noise.

It's probably pointless to mention again that an official PPA would solve that for everybody.

As much as I sympathize with your desire, there is nothing we can do. As mentioned above, people are more than welcome to step up and create a community PPA. But realize that it is a lot of work.

In my view and experience, it's best to work directly together with the distributions. Did you reach out to the Ubuntu community/Canonical and asked whether/how Podman can be updated for LTS versions?

[afbjorklund]

The LTS versions (just like EL) are supposed to ship with old versions, that is sort of their selling point ?

So normally you would have to use something like "backports" (EPEL), to get the latest and greatest...

Another possibility is doing a 3.5 maintenance release.

Or at least getting the deb version updated, from 3.4.2.

[vrothberg]

The LTS versions (just like EL) are supposed to ship with old versions, that is sort of their selling point ?

I think that depends on the individual needs. This is also why there are streams in RHEL for shipping different versions of the container tools.

AFAIK, Ubuntu updates the kernel for LTS versions at some point as well. The balance of "stability" and "new features" is delicate and maintenance for such a long time is a herculean task.

[afbjorklund]

I guess that is now up to the Ubuntu maintainers, how they want to handle the "podman" package... Stay at 3.4 or go to 4.x

Still somewhat ironic that it is easier to upgrade Podman on Mac and on Win, but I guess you can use machine on Linux too.

[lsm5]

It's probably pointless to mention again that an official PPA would solve that for everybody. wink

@alexanderadam I have an UNSTABLE Kubic repo at https://build.opensuse.org/project/show/devel:kubic:libcontainers:unstable . For the near future, it won't get any more official than that.

This repo gets the latest upstream podman and other packages soon after upstream release. I did try it myself sometime ago and AFAICT, 22.04 should mostly just work.

Not recommended for production use, so please be careful where you use it. Let me know what your experience is like.

UPDATE: I'm checking with github actions people about using this repo in their environments and will also update the podman installation docs once I'm certain this works reasonably well.

[Korijn]

@lsm5 Let me just get out of the way that I completely understand your motivations and reasons not to provide a stable PPA. It's a big piece of work and time is a limited resource.

I just want to make sure you are aware that this is also blocking public CI environments from adoption of podman v4. Github actions, azure pipelines, etc are all also stuck.

[lsm5]

@lsm5 Let me just get out of the way that I completely understand your motivations and reasons not to provide a stable PPA. It's a big piece of work and time is a limited resource.

I just want to make sure you are aware that this is also blocking public CI environments from adoption of podman v4. Github actions, azure pipelines, etc are all also stuck.

Yes, see: https://github.com/actions/runner-images/issues/5578 . I'm hoping to make this new kubic repo usable by github actions, but distractions galore.

If anyone would like to help speed this up, do let me know.

[Korijn]

If anyone would like to help speed this up, do let me know.

How can we help?

[lsm5]

How can we help?

@Korijn if you're mainly looking at 22.04, please try using the unstable kubic repo mentioned above and report issues to me. I mainly need end user testing for 22.04 as there are no build / installation / basic usage issues to my knowledge, apart from an annoying Maintainer field warning.

For 20.04 and others, if you look at the referenced github actions issue, there are some glibc related issues. I haven't had the chance to take a good look at those, but if you could dumb those down for me and / or send me patches which can be included in the packaging source, that'd be awesome. I can give you further details later about how the fedora sources build ubuntu packages if you're interested.

[Korijn]

I can definitely try.

... the packaging source, ...

Just checking, is this the right URL to get the packaging sources? https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/unstable/xUbuntu_20.04/

[lsm5]

Just checking, is this the right URL to get the packaging sources? https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/unstable/xUbuntu_20.04/

@Korijn That link contains the release artifacts AFAICT.

For packaging source, you can begin with: https://build.opensuse.org/project/show/devel:kubic:libcontainers:unstable which has the list of packages and build results.

Navigate to one of the packages, for example podman, it will have a _service file with link to the git repo and the branch from where the sources are pulled for building. https://build.opensuse.org/package/view_file/devel:kubic:libcontainers:unstable/podman/_service?expand=1

<services>
  <service name="obs_scm">
    <param name="url">https://gitlab.com/rhcontainerbot/rpms-openqa/podman.git</param>
    <param name="scm">git</param>
    <param name="revision">rawhide</param>
    <param name="extract">podman.spec</param>
  </service>
  <service name="download_files">
    <param name="enforceupstream">yes</param>
  </service>
</services>

The files listed with <param name="extract"> are extracted from that git repo and get used by OBS.

That combined with the debbuild project repository as a dependency added to: https://build.opensuse.org/projects/devel:kubic:libcontainers:unstable/prjconf help to build the deb packages.

Also, let's create a separate issue (either this github or on https://gitlab.com/rhcontainerbot/rpms-openqa/podman.git to continue this conversation and reference that issue here.

[afbjorklund]

Is anyone building 3.4.x ? (20.04)

Could use that for cri-o in minikube

[lsm5]

Is anyone building 3.4.x ? (20.04)

Could use that for cri-o in minikube

does that need to depend on 3.4.x ? /cc @haircommander

[afbjorklund]

No, just has less requirements (and uses cni). Just thought it would be a good idea to get off 3.4.2, without the 4.x breakage...

[lsm5]

No, just has less requirements (and uses cni). Just thought it would be a good idea to get off 3.4.2, without the 4.x breakage...

@afbjorklund RE: cni, is the crio / minikube requirement something that netavark cannot address?

[afbjorklund]

As a Kubernetes node, it will probably use CNI for the foreseeable future

Double network stacks is a possibility but not really a wish - maybe host net ?

[lsm5]

@afbjorklund btw, podman 4.x should still be usable with cni. We can probably adjust dependencies to fetch/ depend on cni for that case.

[afbjorklund]

I might take a look at it, after upgrading to Ubuntu 22.04. Then again, it will probably be Podman 5.0 by then... Considering to go over to the Docker (compat) socket, to get some backwards compatibility.

[lsm5]

I might take a look at it, after upgrading to Ubuntu 22.04. Then again, it will probably be Podman 5.0 by then... Considering to go over to the Docker (compat) socket, to get some backwards compatibility.

alright, in that case I'll wait to hear from you before I go about adjusting deps.

[StarpTech]

Hello, what's the status here? Is there really no default way to install the latest version of podman on the most popular Linux distros? I know that PPA is a pain, but a shell script would do it as well. Or what about https://github.com/NixOS/nix or brew that is available to Linux and Mac.

[lsm5]

Hello, what's the status here? Is there really no default way to install the latest version of podman on the most popular Linux distros? I know that PPA is a pain, but a shell script would do it as well. Or what about https://github.com/NixOS/nix or brew that is available to Linux and Mac.

status is still https://github.com/containers/podman/issues/14302#issuecomment-1192617223

[lsm5]

@Korijn looks like github actions is happy with the version in ubuntu's default repos, so it's unlikely the kubic repo with the latest version will make it there https://github.com/actions/runner-images/issues/5578#issuecomment-1223843419 . Not sure where the azure pipelines people do things but I won't be surprised if they share that opinion too.

Nonetheless, I will update the podman installation docs with the unstable Kubic repo for non-production use soon.

[StarpTech]

status is still https://github.com/containers/podman/issues/14302#issuecomment-1192617223

This looks like an unofficial fork. I'd like to see something that is integrated with the official release flow. I need podman for production use cases.

[Korijn]

@Korijn looks like github actions is happy with the version in ubuntu's default repos, so it's unlikely the kubic repo with the latest version will make it there actions/runner-images#5578 (comment) . Not sure where the azure pipelines people do things but I won't be surprised if they share that opinion too.

They're the same people. Microsoft acquired github a while ago, you may have heard. :) Github actions and azure pipelines use the exact same images.

Nonetheless, I will update the podman installation docs with the unstable Kubic repo for non-production use soon.

Awesome!

[lsm5]

status is still #14302 (comment)

This looks like an unofficial fork. I'd like to see something that is integrated with the official release flow. I need podman for production use cases.

@StarpTech Sure, unofficial as far as the distro is concerned, but as official as it's gonna get from Podman. Maintained by a podman team member (a.k.a me) on company time. That repo should mostly get the same builds that Fedora does at the same speed as Fedora.

The official debian and ubuntu packages are maintained by volunteers in their spare time, and they could use additional hands and eyes for faster update delivery. The podman team doesn't control those, but I can get you in touch with the right people if you'd be willing to help out.

[lsm5]

@StarpTech another option could be download the latest builds from official debian experimental repos. Those often have the latest official distro packages, and the binaries being static should work on other deb / ubuntu versions.

[vrothberg]

status is still #14302 (comment)

This looks like an unofficial fork. I'd like to see something that is integrated with the official release flow. I need podman for production use cases.

There are alternatives such as using another distribution that satisfies the requirements.

I would generally not recommend using "old" distributions and expect modern container tools to "just work". Podman as shipped in Ubuntu works and is rock solid and gets updated with each Ubuntu update. But there is usually a price tag attached if you desire a stable distribution and continuous updates to (container) tools. That's what enterprise distributions are doing and paid for, in particular for production use cases.

I fully empathize with the desire and the request but I don't find it productive to re-iterate over the same conversation. I think it's much better to raise the issue of shipping new Podmans to old Ubuntus to the Ubuntu community. They're doing a great job and maybe Canonical invests resources.

[StarpTech]

I totally understand. It's just super sad from the contributor's perspective as well. I use Debian, and I can't use Podman 4.2 because it's not available on any channel.

That's what enterprise distributions are doing and paid for, in particular for production use cases.

Didn't know. Which company provides paid distributions for Podman on Ubuntu?

I fully empathize with the desire and the request but I don't find it productive to re-iterate over the same conversation. I think it's much better to raise the issue of shipping new Podmans to old Ubuntus to the Ubuntu community. They're doing a great job and maybe Canonical invests resources.

Sounds like you have an idea of how to do this. Please explain.

[lsm5]

I totally understand. It's just super sad from the contributor's perspective as well. I use Debian, and I can't use Podman 4.2 because it's not available on any channel.

For Debian 11, I'm checking with the admins of OBS, the place where the repo I mentioned earlier is hosted. There are some dependency issues preventing a successful build currently.

That's what enterprise distributions are doing and paid for, in particular for production use cases.

Didn't know. Which company provides paid distributions for Podman on Ubuntu?

No paid support on Ubuntu, it's entirely volunteer-based support there. Enterprise support is available for Podman on RHEL. There are no other paid alternatives (to my knowledge at least).

Sounds like you have an idea of how to do this. Please explain.

First step would be to file an issue on ubuntu / debian's bug tracker. The official package maintainer on both distros is @siretart who can provide you with further insight into the debian process. Again, he's generously volunteering his spare time toward this, and combined with debian's own constraints, there will be limits on what can be shipped in the official repositories.

Hope that helps.

[TCROC]

I believe I mentioned it earlier and it was said the community could do it if they wanted, but I just thought I'd bring it up again:

Why is using a cross platform packaging software such as Snapcraft as an official distro not being discussed more? I believe the selling point of these packaging tools is they are supposed to solve a lot of these complexities. Is there a reason Podman wouldn't want to officially use one?

^ Keep in mind, I'm merely a naive dev who wants to use the shiny toys such as new Podman on my Ubuntu :)

[TCROC]

Ah found it. It got moved to a discussion. Lives over here now :)

https://github.com/containers/podman/discussions/14360