Files copied incorrectly between multistage build

[calebwoofenden]

Actual behavior A clear and concise description of what the bug is.

I have a multi-stage Dockerfile. The first stage, package_installer, installs Poetry and then installs Python packages with Poetry. The second, primary stage copies files from this stage:

COPY --from=package_installer /usr/local/lib/python3.7/site-packages/ /usr/local/lib/python3.7/site-packages/
COPY --from=package_installer /usr/local/bin/ /usr/local/bin/

When running a container from the image built by Kaniko using this Dockerfile, I run into strange Python errors. I used container-diff to compare this image to one created from the same Dockerfile with docker build and found that an enormous amount of files in /usr/local/lib and /usr/local/bin are changed. Most notably, every .pyc file in Python's site-packages is noted as changed.

Additionally, the image from the same Dockerfile built via Kaniko is about 15% larger than the one built via docker build.

Expected behavior A clear and concise description of what you expected to happen.

The image should be the same whether it's built with Kaniko or docker build.

To Reproduce Steps to reproduce the behavior:

1. Create a Dockerfile with Python installed that installs Poetry and then installs some packages with Poetry
2. Make it multi-stage and copy the above COPY commands in the second stage, being sure to change to the appropriate Python version

Additional Information

• Dockerfile I cannot provide this exact Dockerfile but I can provide more detail if there is some missing.
• Build Context Built in a k8s Gitlab runner pod using the latest Kaniko image

• Kaniko Image (fully qualified with digest) This happens on the latest version of the official Kaniko image (1.8.1) and also on 1.7.0.

  Triage Notes for the Maintainers

  Description	Yes/No
  Please check if this a new feature you are proposing	- [ ]
  Please check if the build works in docker but not in kaniko	- [x]
  Please check if this error is seen when you use --cache flag	- [ ]
  Please check if your dockerfile is a multistage dockerfile	- [x]

[henricook]

This is going to be very hard for anyone to work on if they have to come up with the reproduction case themselves. If you'd like it fixing I suggest you provide the minimal dockerfiles to recreate the problem here.