search

frerich / clcache

A compiler cache for MSVC, much like ccache for gcc
Other
325 stars 83 forks source link

ClCache mess up with MSBuild cache because of STATS.TXT #355

Closed NicolasDorier closed 5 years ago

NicolasDorier commented 5 years ago

When rebuilding a project without ClCache without change, it takes 15 seconds, but with ClCache, several minutes.

The reason is that the CL task can be skipped if all inputs access time are inferior to outputs access time (using tlog under the hood).

When compiling with ClCache, here is my logs:

Task "CL"
  Read Tracking Logs:
        x64\Debug\libbitcoinconsensus\libbitco.2B384FA8.tlog\clcache.read.1.tlog
  Outputs for C:\SOURCES\BITCOIN\SRC\ARITH_UINT256.CPP|C:\SOURCES\BITCOIN\SRC\CONSENSUS\MERKLE.CPP|C:\SOURCES\BITCOIN\SRC\CONSENSUS\TX_CHECK.CPP|C:\SOURCES\BITCOIN\SRC\CRYPTO\AES.CPP|C:\SOU
  RCES\BITCOIN\SRC\CRYPTO\CHACHA20.CPP|C:\SOURCES\BITCOIN\SRC\CRYPTO\HMAC_SHA256.CPP|C:\SOURCES\BITCOIN\SRC\CRYPTO\HMAC_SHA512.CPP|C:\SOURCES\BITCOIN\SRC\CRYPTO\RIPEMD160.CPP|C:\SOURCES\BIT
  COIN\SRC\CRYPTO\SHA1.CPP|C:\SOURCES\BITCOIN\SRC\CRYPTO\SHA256.CPP|C:\SOURCES\BITCOIN\SRC\CRYPTO\SHA256_SSE4.CPP|C:\SOURCES\BITCOIN\SRC\CRYPTO\SHA512.CPP|C:\SOURCES\BITCOIN\SRC\HASH.CPP|C:
  \SOURCES\BITCOIN\SRC\PRIMITIVES\BLOCK.CPP|C:\SOURCES\BITCOIN\SRC\PRIMITIVES\TRANSACTION.CPP|C:\SOURCES\BITCOIN\SRC\PUBKEY.CPP|C:\SOURCES\BITCOIN\SRC\SCRIPT\BITCOINCONSENSUS.CPP|C:\SOURCES
  \BITCOIN\SRC\SCRIPT\INTERPRETER.CPP|C:\SOURCES\BITCOIN\SRC\SCRIPT\SCRIPT.CPP|C:\SOURCES\BITCOIN\SRC\SCRIPT\SCRIPT_ERROR.CPP|C:\SOURCES\BITCOIN\SRC\UINT256.CPP|C:\SOURCES\BITCOIN\SRC\UTIL\
  STRENCODINGS.CPP:
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\ARITH_UINT256.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\MERKLE.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\TX_CHECK.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\AES.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\CHACHA20.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\HMAC_SHA256.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\HMAC_SHA512.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\RIPEMD160.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\SHA1.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\SHA256.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\SHA256_SSE4.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\SHA512.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\HASH.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\BLOCK.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\TRANSACTION.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\PUBKEY.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\BITCOINCONSENSUS.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\INTERPRETER.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\SCRIPT.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\SCRIPT_ERROR.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\UINT256.OBJ
        C:\SOURCES\BITCOIN\BUILD_MSVC\LIBBITCOINCONSENSUS\X64\DEBUG\LIBBITCOINCONSENSUS\STRENCODINGS.OBJ
  ..\..\src\arith_uint256.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
  ..\..\src\consensus\merkle.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
  ..\..\src\consensus\tx_check.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
  ..\..\src\crypto\aes.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
  ..\..\src\crypto\chacha20.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
  ..\..\src\crypto\hmac_sha256.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
  ..\..\src\crypto\hmac_sha512.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
  ..\..\src\crypto\ripemd160.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
  ..\..\src\crypto\sha1.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
  ..\..\src\crypto\sha256.cpp will be compiled because C:\USERS\NICOLASDORIER\CLCACHE\STATS.TXT was modified at 7/1/2019 10:27:27 PM.
NicolasDorier commented 5 years ago

Solution:

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <CLToolExe>clcache.exe</CLToolExe>
    <CLToolPath>C:\ProgramData\chocolatey\bin\</CLToolPath>
    <NoDependencies>$(USERPROFILE)\clcache\stats.txt</NoDependencies>
  </PropertyGroup>
</Project>

NoDependencies makes sure that CL does not check the stats file.

NicolasDorier commented 5 years ago

Ping @frerich I think it can be nice to document this somewhere in the doc if you have time ;)