SkipUnchangedFiles doesn't work on Linux

marcin-krystianc commented 5 years ago

On Windows and MacOS File.Copy preserves LastWriteTime, but it is not the case on Linux. For this reason SkipUnchangedFiles=true doesn't really work on Linux, because we use LastWriteTime to check if file has changed or not. Linux doesn't preserve LastWriteTime, so it will always differ for original file and copy.

https://github.com/Microsoft/msbuild/pull/3997/files#diff-4826624e1418003ab55be4305eadddadR568

rainersigwald commented 5 years ago

Do you know if the timestamp is really not preserved, or if it's preserved incorrectly? This could be a symptom of https://github.com/dotnet/corefx/issues/31379, which is fixed for 3.0. If that's the case, I'd rather not take an MSBuild fix since it'll be inefficient after getting the runtime fix.

marcin-krystianc commented 5 years ago

Do you know if the timestamp is really not preserved, or if it's preserved incorrectly? This could be a symptom of dotnet/corefx#31379, which is fixed for 3.0. If that's the case, I'd rather not take an MSBuild fix since it'll be inefficient after getting the runtime fix.

I've opened this ticket, because CI pipeline for msbuild reports that SkipUnchangedFiles doesn't work on Linux (See the workaround that is necessary in this test https://github.com/Microsoft/msbuild/blob/95862314f14733dd0086b1747264c40e6989f2cf/src/Tasks.UnitTests/Copy_Tests.cs#L568) I wrote a netcore2.1 app to reproduce it and run it on Win10, real Ubuntu18.04 and Ubuntu18.04/WSL. So it seems that File.Copy and SetLastWriteTime work differently on Win10, real Ubuntu18.04 and Ubuntu18.04/WSL.

Win10 -> File.Copy preserves timestamp and SetLastWriteTime doesn't loose precision
Ubuntu18.04 -> File.Copy preserves timestamp and SetLastWriteTime looses precision (it is rounded to seconds dotnet/corefx#31379)
Ubuntu18.04/WSL -> File.Copy doesn't preserve timestamps (very small difference in number of ticks) and SetLastWriteTime looses precision (rounded to seconds)

From these results we see that only on Ubuntu18.04/WSL timestamps are not preserved for copies. Therefore we can conclude that the actual issue here is that WSL has a bug and CI pipeline for msbuild doesn't run on real Linux, but it rather runs on Linux/WSL.

TestApp:

    class Program
    {
        static void Main(string[] args)
        {
            var files = new[] { "some_file.0", "some_file.1", "some_file.2", "some_file.3" };
            foreach (var f in files)
                File.Delete(f);

            File.WriteAllText(files[0], "");
            Thread.Sleep(2000);
            File.Copy(files[0], files[1], true);
            Thread.Sleep(2000);
            File.Copy(files[1], files[2], true);
            File.WriteAllText(files[3], "");
            File.SetCreationTimeUtc(files[3], File.GetCreationTimeUtc(files[0]));
            File.SetLastWriteTimeUtc(files[3], File.GetLastWriteTimeUtc(files[0]));

            var creationTimes = files.Select(f => File.GetCreationTimeUtc(f)).ToArray();
            var writeTimes = files.Select(f => File.GetLastWriteTimeUtc(f)).ToArray();

            Console.WriteLine($" file_name\t cration_time\t\t last_write_time\t");

            for (var i = 0; i < files.Length; i++)
            {
                Console.WriteLine($" {files[i]}\t {creationTimes[i].Ticks}\t {writeTimes[i].Ticks}\t");
            }
        }
    }

Win10

file_name       creation_time            last_write_time
 some_file.0     636807631884239034      636807658196795794
 some_file.1     636807631904280965      636807658196795794
 some_file.2     636807631924320285      636807658196795794
 some_file.3     636807631884239034      636807658196795794

Ubuntu18.04

file_name    creation_time       last_write_time    
 some_file.0     636807656652480010  636807656652480010 
 some_file.1     636807656652480010  636807656652480010 
 some_file.2     636807656652480010  636807656652480010 
 some_file.3     636807656650000000  636807656650000000

Ubuntu18.04/WSL

file_name       creation_time            last_write_time
 some_file.0     636807648584019239      636807648584019239
 some_file.1     636807648584019230      636807648584019230
 some_file.2     636807648584019230      636807648584019230
 some_file.3     636807648580000000      636807648580000000

dotnet / msbuild

SkipUnchangedFiles doesn't work on Linux #4015