Option to suppress Julia re-precompiling a package under development

[staticfloat]

When working on packages that involve compiler internals, it's often useful to have Revise open and running for ease of experimentation, but also need to constantly kill the Julia session due to caching effects within the compiler. When the new Julia session attempts to load BigCompilerPackage, it will see that there have been changes made to the source code and then embark upon a massive precompilation undertaking, but I already have Revise loaded, so I'd much rather load the old precompile cache, and then have Revise monkey-patch in the changed definitions.

Would it be possible to have a mode where Revise can suppress the traditional Julia precompilation check, and manually update the loaded module according to the modified source on-disk? Perhaps by comparing timestamps of the files on-disk against the precompilation cache file timestamp?

[timholy]

That's a pretty interesting idea. It's a keeper.

[aviatesk]

So the idea is to accomplish something similar to what we can do with LocalPreferences.toml and the skip_precompile setting, but automatically? Or is something more advanced wanted here?

[staticfloat]

No, this is separate. The idea is to avoid Julia’s builtin compilation trigger when loading a module.

[timholy]

Assuming I understand it correctly, it's to by-pass the "is the cachefile stale?" logic for certain specified packages: load the cachefile despite its being stale, and then bring the code up-to-date with Revise. In other words, it mimicks what happens during the running session but still allows you to reset the other aspects of your system by restarting Julia.

[KristofferC]

But you will have a lot of stale cachefiles for the same package so you have to pick one of them. Probably the one most recently modified?

[tecosaur]

Perhaps the new version could also be quietly precompiled in the background? To make sure that you don't just get a growing pile of changes from an increasingly-old checkpoint.

[johnomotani]

Perhaps the new version could also be quietly precompiled in the background? To make sure that you don't just get a growing pile of changes from an increasingly-old checkpoint.

Even better would be to actually update the cache file in the same way the compiled code would be updated during a running session!

[timholy]

Even better would be to actually update the cache file in the same way the compiled code would be updated during a running session!

That's really really hard to the point of near-impossibility. We write a "packed" (in memory) pkgimage and doing surgery on it is difficult. I suppose we could "nuke" the old implementation and append new stuff, but there are also pointer cross-references and all those would have to be updated. For 2023 this does not seem worth the amount of effort it would require.

[timholy]

There's a slightly different proposal that would put such a mechanism into Julia itself here.