timholy / SnoopCompile.jl

Provide insights about latency (TTFX) for Julia packages
https://timholy.github.io/SnoopCompile.jl/dev/
Other
309 stars 48 forks source link

Add a copy-paste invalidations analysis script #326

Closed ChrisRackauckas closed 1 year ago

ChrisRackauckas commented 1 year ago

As much as details are nice, sometimes giving something that's easy to copy/repeat is good. Maybe this could be added as a library function (it probably would need to be a macro).

ChrisRackauckas commented 1 year ago

I'll go by your recommendations, but I think my main points are:

  1. I think the tutorial should have a simple copy paste "what are my invalidations?" before the deeper explanation of the tool
  2. It should also produce a plot to show the relative scale of which invalidations matter.

So I updated to make use of the version from https://timholy.github.io/SnoopCompile.jl/dev/reference/#SnoopCompile.precompile_blockers. For some reason I find that way it's written to be much easier to read than the one in https://discourse.julialang.org/t/julia-v1-9-0-beta2-is-fast/92290/17?u=tim.holy, maybe just due to white space.

For the plots, should I be doing that all on staletrees instead of trees?

ChrisRackauckas commented 1 year ago

Maybe this should be in a separate "Getting Started" or "SnoopCompiler 101 for Inexperienced Programmers" or something. I think the big thing about SnoopCompile is that now with the precompilation changes and all, it needs to move from an audience of "I know a bit of how Julia works and want to make a good package better" to "this is some standard Julia 201 stuff like type-stability". I don't think your average "just started developing packages" developer should need to know how to dig into the details and fix invalidations. I think it's sufficient for many to just know that (a) invalidations are an issue, (b) here's how I create a report of invalidations and open an issue. If people are opening these issues and the big ones are brought into the conversation, then I think they will be solved.

timholy commented 1 year ago

I agree this has to get a lot simpler. Most of the docs were written in the course of this basically being a research project, trying to understand what factors would affect precompilation. Now that Julia itself is much more capable, this is a much less complex question than it used to be. We definitely need to make the documentation reflect the new world.

In fact what I think we also desperately need is a way to take the analysis that Cthulhu does, but instead of annotating the type-inferred code, instead annotates the raw source text. E.g.,

    for (i, x::Any) in enumerate(v)
        g[i] = f(x)::Any
    end

but making that happen is, I think, quite difficult. https://github.com/JuliaLang/julia/issues/31162 would make this vastly easier, but I don't know lisp and have no intention of learning.

timholy commented 1 year ago

I think we can go with this, and add other forms of plotting later. Thanks @ChrisRackauckas! I'll merge when CI finishes.

codecov[bot] commented 1 year ago

Codecov Report

Base: 84.08% // Head: 77.04% // Decreases project coverage by -7.04% :warning:

Coverage data is based on head (7a6aa05) compared to base (dcae62d). Patch has no changes to coverable lines.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #326 +/- ## ========================================== - Coverage 84.08% 77.04% -7.05% ========================================== Files 17 17 Lines 2162 2021 -141 ========================================== - Hits 1818 1557 -261 - Misses 344 464 +120 ``` | [Impacted Files](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy) | Coverage Δ | | |---|---|---| | [src/invalidation\_and\_inference.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-c3JjL2ludmFsaWRhdGlvbl9hbmRfaW5mZXJlbmNlLmps) | `0.00% <0.00%> (-89.66%)` | :arrow_down: | | [src/invalidations.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-c3JjL2ludmFsaWRhdGlvbnMuamw=) | `63.90% <0.00%> (-13.90%)` | :arrow_down: | | [src/parcel\_snoopi\_deep.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-c3JjL3BhcmNlbF9zbm9vcGlfZGVlcC5qbA==) | `86.32% <0.00%> (-3.58%)` | :arrow_down: | | [SnoopPrecompile/src/SnoopPrecompile.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-U25vb3BQcmVjb21waWxlL3NyYy9Tbm9vcFByZWNvbXBpbGUuamw=) | `85.71% <0.00%> (-1.79%)` | :arrow_down: | | [src/write.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-c3JjL3dyaXRlLmps) | `90.90% <0.00%> (-1.40%)` | :arrow_down: | | [src/deep\_demos.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-c3JjL2RlZXBfZGVtb3Muamw=) | `57.89% <0.00%> (-1.08%)` | :arrow_down: | | [src/parcel\_snoopl.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-c3JjL3BhcmNlbF9zbm9vcGwuamw=) | `89.47% <0.00%> (-0.53%)` | :arrow_down: | | [src/parcel\_snoopi.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-c3JjL3BhcmNlbF9zbm9vcGkuamw=) | `93.83% <0.00%> (-0.31%)` | :arrow_down: | | [src/visualizations.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-c3JjL3Zpc3VhbGl6YXRpb25zLmps) | `0.00% <0.00%> (ø)` | | | [src/parcel\_snoopc.jl](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy#diff-c3JjL3BhcmNlbF9zbm9vcGMuamw=) | `78.37% <0.00%> (+0.22%)` | :arrow_up: | | ... and [1 more](https://codecov.io/gh/timholy/SnoopCompile.jl/pull/326/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy) | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tim+Holy)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.