img pull distribution manifest is different from docker pull

[mayank-agarwal-96]

When I do img pull busybox and img ls, the digest is sha256:05313277ba61c604f48dbe9915100d47f6b6a1d2051a82bd0590b6372e91f8a9 while when I do docker pull, the digest is sha:f79f7a10302c402c052973e3fa42be0344ae6453245669783a9e16da3d56d5b4. Both manifests are present in /.local/share/img/runc/native/content/blobs/sha256. They each point to different config file. The one img uses misses on many fields including path, environment variables exported in the Docker image.

~/.local/share/img/runc/native/content/blobs/sha256 # cat f79f7a10302c402c052973e3fa42be0344ae6453245669783a9e16da3d56d5b4 | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "size": 1497,
    "digest": "sha256:af2f74c517aac1d26793a6ed05ff45b299a037e1a9eefeae5eacda133e70a825"
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 755841,
      "digest": "sha256:fc1a6b909f82ce4b72204198d49de3aaf757b3ab2bb823cb6e47c416b97c5985"
    }
  ]
}
~/.local/share/img/runc/native/content/blobs/sha256 # cat 05313277ba61c604f48dbe9915100d47f6b6a1d2051a82bd0590b6372e91f8a9 | jq
{
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "digest": "sha256:aa0db434be971b1082ede49cbeda6fc279afda4403a222eff94894bcac66009f",
    "size": 450
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "digest": "sha256:fc1a6b909f82ce4b72204198d49de3aaf757b3ab2bb823cb6e47c416b97c5985",
      "size": 755841
    }
  ]
}

Contents of the two digest:

~/.local/share/img/runc/native/content/blobs/sha256 # cat aa0db434be971b1082ede49cbeda6fc279afda4403a222eff94894bcac66009f | jq .
{
  "architecture": "amd64",
  "config": {
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "WorkingDir": "/"
  },
  "created": "2019-04-26T09:58:22.9468038Z",
  "history": [
    {
      "created": "2019-04-26T09:58:22.9468038Z",
      "created_by": "pulled from docker.io/library/busybox:latest",
      "comment": "buildkit.exporter.image.v0"
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:0b97b1c81a3200e9eeb87f17a5d25a50791a16fa08fc41eb94ad15f26516ccea"
    ]
  }
}
~/.local/share/img/runc/native/content/blobs/sha256 # cat af2f74c517aac1d26793a6ed05ff45b299a037e1a9eefeae5eacda133e70a825 | jq .
{
  "architecture": "amd64",
  "config": {
    "Hostname": "",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "sh"
    ],
    "ArgsEscaped": true,
    "Image": "sha256:90b7037cc5e65fa9e3f33e8096febd6fad8af0ff94876d73dabe048d65bec645",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": null
  },
  "container": "2639c6a03f9dec9b8a4563ff4572f3f0a2a2027fc2801e8b10a0b2fd6825ded8",
  "container_config": {
    "Hostname": "2639c6a03f9d",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "/bin/sh",
      "-c",
      "#(nop) ",
      "CMD [\"sh\"]"
    ],
    "ArgsEscaped": true,
    "Image": "sha256:90b7037cc5e65fa9e3f33e8096febd6fad8af0ff94876d73dabe048d65bec645",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": {}
  },
  "created": "2019-04-02T23:32:10.727183061Z",
  "docker_version": "18.06.1-ce",
  "history": [
    {
      "created": "2019-04-02T23:32:10.579876415Z",
      "created_by": "/bin/sh -c #(nop) ADD file:6051b0ebe4098ccbb14f63db9ae7e32fa8ed55b5de34bc43399e37b5f12651b6 in / "
    },
    {
      "created": "2019-04-02T23:32:10.727183061Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"sh\"]",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:0b97b1c81a3200e9eeb87f17a5d25a50791a16fa08fc41eb94ad15f26516ccea"
    ]
  }
}

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.61. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[kekoav]

Duplicate of #199 ?

[pandada8]

I think fix #234 should fix #199 the problem seems caused by a fix to #110 https://github.com/genuinetools/img/commit/b38a9eb63126413a3388f5e642c2f93ef9321908. which create a snapshot when pulling the image.

[mitchellh]

I just opened a PR to fix #199 (PR: https://github.com/genuinetools/img/pull/326) but that will not resolve this exactly. The layer digests do match exactly but the root digest does not if the image has Docker config schema and not an OCI config schema. In my PR, on pull, the schema is converted to an OCI schema, so the digest will change but most of the metadata is still retained.

I think my PR is still a good step forward cause it gets pull retaining any config. I'm not sure how important it is to folks to retain the identical Docker schemas...