Provide convenience utilities for invalidation

[timholy]

https://github.com/JuliaLang/julia/pull/35729 will make it possible to determine the causes of invalidation. It would be nice to organize these reports in a way that makes them more digestible to the user. It should be a fairly simple parsing problem, along with managing the state of the debug flag.

Helping users interpret the cause & cure might be more challenging; the kind of analysis in https://github.com/timholy/Revise.jl/issues/456#issuecomment-623024569 requires knowledge of many different aspect of Julia. Not sure what besides documentation one can do here.

@vtjnash, what do you think should be done with the -- isequal(Method, Method) lines? I'm inclined to drop them if they don't trigger a non--- invalidation, but perhaps I don't appreciate their full significance.

[aminya]

It would be nice if we merge https://github.com/timholy/SnoopCompile.jl/pull/71 before adding new features.

[timholy]

Yep. Julia 1.5 feature freeze arrives at the end of today, so I'll be more available shortly.

[IanButterworth]

I was thinking about ways to report this in a github action. Perhaps:

• Trigger with@SnoopCompileBot checkinvalidations "using Package"
• Report total count and list of invalidations (">>" lines) with number of re-compiled calls per invalidation as a bot comment, with a link to the full text dump

Once https://github.com/JuliaLang/julia/pull/35729 lands in a nightly, I think I'll try it out directly and then swap in any SnoopCompile support when added

[aminya]

@timholy Should I implement anything regarding the invalidations into the SnoopCompile bot?

[timholy]

That probably makes sense, though it's not entirely obvious what to do here. When loading PkgA triggers invalidations, it's only occasionally because PkgA defined a method that it shouldn't have defined. Most often it's because its dependencies (Base, StdLibB, PkgC, etc) need re-engineering. So I guess the key question is, who should get this info? The author of PkgA might be interested in why so little of PkgA is precompilable (due to all those invalidations), but the fixes typically lie elsewhere.

[timholy]

Update: last piece of the puzzle is https://github.com/JuliaDebug/Cthulhu.jl/pull/82, so we can close this when that merges.

[aminya]

We can add another step after the benchmark step to upload the invalidation data for anyone interested.

[timholy]

Yeah, I think that makes sense. In the end package authors will probably want to know this information.

How detailed should the report be? A very minimal summary is the one-liner in https://github.com/timholy/SnoopCompile.jl/blob/master/examples/invalidations_blog.jl. A list of the method definitions in the package triggering invalidation and the total number of invalidations they trigger seems likely to be more useful, though. The full print-out from invalidation_trees (see example at https://timholy.github.io/SnoopCompile.jl/stable/snoopr/#Recording-invalidations-1) provides even more detail.

[aminya]

After we merged #98 and did the inner registrations:

• [ ] we can try to add a function in SnoopCompileBot (like snoop_invalidations), which automates the correct workflow for snooping of invalidations. We will use the same BotConfig and example_script, and similar to snoop_bot and snoop_bench, it allows us to do this on multiple operating systems and Julia versions.

• [ ] Bot should write the trees to an HTML file (to preserve the colors of printstyled).

• [ ] In the HTML report, we can also add links to the actual source code in the repository.