Closed tlnagy closed 8 years ago
Pkg check staleness
staleness check happens in using
(actually require
, which is also used by import
), not Pkg
Thanks! I updated the original post.
I see, the problem here is that the timestamp has changed because Pkg.build("Rmath")
regenerates its deps.jl
file, but the code hasn't actually changed so the rebuild (which triggers #12508) is not actually needed.
As a workaround, @BinDeps.install
could overwrite the deps.jl
file only if it has changed.
that was a workaround I suggested in the StatsBase issue, but I think this could happen for other reasons and might be good for making .ji
files easier to redistribute, if the checksum is cheap enough. x-ref https://github.com/JuliaLang/julia/pull/12458#issuecomment-129859137
Making staleness depend on the hash would be great for users who work on distributed system (supercomputers and such). @andreasnoack and I have been running into problems sometimes.
Here is a nice-looking hardware-accelerated BSD-licensed CRC32c implementation in C by Mark Adler: http://stackoverflow.com/questions/17645167/implementing-sse-4-2s-crc32c-in-software/1
And here is a benchmark of different CRC routines, concluding that Adler's routine is pretty much the fastest.
I've always been fairly against doing this with checksums. I agree there needs to be more development here, but I haven't been able to develop a complete proposal yet to address these and other similar issues.
@vtjnash, doing if !timestamps_match && !checksums_match then regenerate_cache
seems like a strict improvement over if !timestamps_match
(if the time for the checksum is negligible, which seems likely), by reducing the chance of false positives. What is the nature of your objection?
Should this be closed in light of the discussion at #18127?
As discussed in https://github.com/JuliaStats/StatsBase.jl/issues/202, alternating
Pkg.add(X)
andusing X
statements can lead to errors. According to @tkelman, this could be fixed by makingrequire
check staleness by looking at the timestamp instead of a hash.Example: