RFC: Example Rule "Gallery" to showcase CRC best practices?

[doddgray]

Following up on some Slack (#autodiff) discussion with @mzgubic, I want to float the idea of adding a CRC rule "gallery" to the ChainRules docs, inspired by analogous things in the plotting world (eg BeautifulMakie).

I think that best practices for rule authors have become a bit too complicated to sufficiently demonstrate with just one or two small examples per section in the docs, particularly with interplay between

• custom structs & their constructors
• higher order functions
• CRC Thunks, Tangent types & AD callbacks

There are many clever rule authors in the wild who post helpful examples on Github/Discourse/Slack/Zulip but this information is very disaggregated and un-searchable. If there were some way for users to submit nice but less ultra-minimal examples and/or to aggregate relevant examples of custom rrule and frule definitions in published packages (as @ToucheSir suggested) I think linking to such a "gallery" from the CR docs would speed up the learning process for rule authors.

I would be glad to help realize such a thing if it seemed feasible and people wanted it.

Less organized thoughts from Slack: A few custom rule examples I can find in forums but which I wouldn't know how to write based on CR docs:

• Constructors of widely used AbstractArray subtypes like StaticArrays, StructArrays, etc.
• LinearAlgebra methods of “implicit operator” types representing giant Matrices, eg from LinearMaps.jl
• Iterative solver and partial decomposition methods involving the former, eg methods from IterativeSolvers.jl, KrylovKit.jl, Arpack.jl, etc, (“adjoint method” VJP rules)
• Higher order function methods defined outside Base, often versions of map or reduce with parallelism.
• Gradient accumulation into an InplaceableThunk of a custom struct

[oxinabox]

Perhaps we can do this in a way that just retrieves and shows the right code loaded from straight from github?

[doddgray]

That would be great. If you'd be open to that kind of addition I can look around for examples and try to get a pull request together. That would go to ChainRules.jl right?

[mcabbott]

It's a bit of a moving target, as we try to figure out the best practice. So I guess that pointing out which bits of ChainRules's source to look at might be better than writing code specifically about this, less likely to end up outdated? Perhaps the docs can guide you a bit as to what the various parts are doing, or (for instance) how the simplest version would work, before optimisations / care about 2nd derivatives adds complication.

[oxinabox]

That would be great. If you'd be open to that kind of addition I can look around for examples and try to get a pull request together. That would go to ChainRules.jl right?

Docs go in ChainRulesCore.jl (this repo)

I think the code for live loading should like like this code in the CROG.jl docs Something like:

````@eval
using Download
using Markdown
Markdown.parse("""
```julia
$(read(Download.download("https://github.com/JuliaDiff/ChainRules.jl/blob/main/src/rulesets/LinearAlgebra/structured.jl#L210-L215"), String))

""")

One issue with this, as I think @mcabbott is pointing out is that as files change those line numbers might move