"Image not known" when using Arion with Podman on Ubuntu

[GWspekulatius]

Hi, i am trying to use the minimal webserver from documentation arion-compose.nix:

{ pkgs, ... }:
{
  config.services = {

    webserver = {
      service.useHostStore = true;
      service.command = [
        "sh"
        "-c"
        ''
          cd "$$WEB_ROOT"
          ${pkgs.python3}/bin/python -m http.server
        ''
      ];
      service.ports = [
        "8000:8000" # host:containerA
      ];
      service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
    };
  };
}

arion-pkgs.nix:

import <nixpkgs> { system = "x86_64-linux"; }

with Arion and Podman 3.2.2. on Ubuntu 20.04., getting this error when doing arion up:

these derivations will be built:
  /nix/store/i8jr1ka9hifc7kmkk0aq3amcxdpj9m51-webserver-base.json.drv
  /nix/store/psxpl4scln67783j6s9y0x8h37jwvzjd-closure.drv
  /nix/store/vzyddm07qi8fxvcx1zkjpzk1p6h9ssc8-webserver-customisation-layer.drv
  /nix/store/vw9mni5wdj888k47r0p6ygdiya5cnm6m-webserver-conf.json.drv
  /nix/store/v5d8rdijfs4pb43y1n11d3sfwy7d5ipv-stream-webserver.drv
  /nix/store/mjgn4hz95h51caxkxmdpdpwvmbb1309z-docker-compose.yaml.drv
these paths will be fetched (0.00 MiB download, 0.00 MiB unpacked):
  /nix/store/himhj7kdg5gd7mnzzxf427927z5rf7wd-closure-paths
  /nix/store/rja1ahifyfvg1fnjxjp3v5zrz53i11mk-closure
copying path '/nix/store/rja1ahifyfvg1fnjxjp3v5zrz53i11mk-closure' from 'https://cache.nixos.org'...
building '/nix/store/i8jr1ka9hifc7kmkk0aq3amcxdpj9m51-webserver-base.json.drv'...
copying path '/nix/store/himhj7kdg5gd7mnzzxf427927z5rf7wd-closure-paths' from 'https://cache.nixos.org'...
building '/nix/store/vzyddm07qi8fxvcx1zkjpzk1p6h9ssc8-webserver-customisation-layer.drv'...
Generating the nix database...
Warning: only the database of the deepest Nix layer is loaded.
         If you want to use nix commands in the container, it would
         be better to only have one layer that contains a nix store.
building '/nix/store/vw9mni5wdj888k47r0p6ygdiya5cnm6m-webserver-conf.json.drv'...
{
  "architecture": "amd64",
  "config": {
    "Cmd": []
  },
  "os": "linux",
  "store_dir": "/nix/store",
  "from_image": null,
  "store_layers": [],
  "customisation_layer": "/nix/store/g8zhwkad802aci2svp4gpjxbmq18px2p-webserver-customisation-layer",
  "repo_tag": "webserver:1qvknidf81kdxf60i71p9jkyjhip8ln7",
  "created": "1970-01-01T00:00:01+00:00"
}
building '/nix/store/v5d8rdijfs4pb43y1n11d3sfwy7d5ipv-stream-webserver.drv'...
building '/nix/store/mjgn4hz95h51caxkxmdpdpwvmbb1309z-docker-compose.yaml.drv'...
/nix/store/bzhqs231milxfkgmydgifpfgrv3lh04h-docker-compose.yaml
No 'fromImage' provided
Creating layer 1 with customisation...
Adding manifests...
Done.
Getting image source signatures
Copying blob fd7fcd30bf5a done  
Copying config d3668222c1 done  
Writing manifest to image destination
Storing signatures
Loaded image(s): localhost/webserver:1qvknidf81kdxf60i71p9jkyjhip8ln7
Creating network "arion_default" with the default driver
Pulling webserver (webserver:1qvknidf81kdxf60i71p9jkyjhip8ln7)...
Creating arion_webserver_1 ... 
Creating arion_webserver_1 ... error

ERROR: for arion_webserver_1  Cannot create container for service webserver: webserver:1qvknidf81kdxf60i71p9jkyjhip8ln7: image not known

ERROR: for webserver  Cannot create container for service webserver: webserver:1qvknidf81kdxf60i71p9jkyjhip8ln7: image not known
ERROR: Encountered errors while bringing up the project.

Anyone has any clue what this means? Is this caused by a false configuration in my system or is this an arion issue?

[roberth]

My first guess is that docker and docker-compose talk to different engines.

Did you enable dockerCompat [in NixOS] instead of virtualisation.podman.dockerSocket.enable by any chance? I don't think I've tested that scenario and I suspect it may break like this.

[GWspekulatius]

Hi Robert, these look like NixOS flags to me. As i am just getting started with my Nix endeavours, i havent made the transition to NixOS yet and am still on normal Ubuntu. so i dont think i can use these flags. So far I have created symbolic links to trick arion into using podman instead of docker. one link for the binary and another one for the unix socket which is what the nix flags do.

[roberth]

I see. Arion talks to the docker and docker-compose on PATH. As long as those agree about the runtime, I wouldn't expect to see a problem like this. Given that it's Ubuntu, it's quite possible that I haven't tested with its versions of podman and friends.

Could you check that docker and docker-compose talk to the same engine?

[GWspekulatius]

i set up a completely new Ubuntu VM, installed only nix, arion, podman and docker-compose (so no docker). so they must be using the podman engine. i do the symlinks and change some permissions but eventually i end up at the described exception where i don't know how to continue.

in case anyone wants to reproduce this on ubuntu 21.04: i installed podman manually via https://build.opensuse.org/package/show/devel:kubic:libcontainers:stable/podman because its a newer version than the one on apt. symlink for the docker binary: ln -s /usr/bin/podman docker (start arion with PATH=./:$PATH arion up) symlink for the docker unix socket: sudo ln -s /run/podman/podman.sock /run/docker.sock add permissions: sudo chmod -R 755 /run/podman/