search

fredrikekre / Literate.jl

Simple package for literate programming in Julia
https://fredrikekre.github.io/Literate.jl
Other
534 stars 61 forks source link

"Makefile" like capability for Literate examples? #226

Open JeffFessler opened 1 year ago

JeffFessler commented 1 year ago

I have a repo with over 30 Literate demos for a textbook, and no source code under src/. https://github.com/JeffFessler/book-mmaj-demo Every time I edit one of the demos, the Literate/Documenter build reruns all 30+ demos, which takes a long time and might be related to my time-out issue #225. Is there a way to set up the repo / yaml files / make.jl so that it behaved more like a Makefile would, where the workflow would only rerun the demos that have changed, instead of all of them? (I've studied the docs for Literate and Documenter and could not find an answer.) Such a capability could reduce a lot of CO2... Ideas?

fredrikekre commented 12 months ago

Are you aware of Documenters draft mode? draft = true to makedocs(...).

JeffFessler commented 12 months ago

Yes, I sometimes use that and expandfirst when I am debugging locally. But then when it is ready I push to github (without those debugging flags) and make a PR and the github actions build and rerun all the demos, most of which have not changed, wasting CPU time in the cloud. Which is why I was asking if there was a way to set up the Documenter.yaml or make.jl or such to be smarter about it in the cloud.

There is one brute-force option I've considered, which is to put every demo in its own repo, because then only one demo at a time would get built by github actions. That would be a maintenance pain though...

Another option might be to have one repo with a separate "subpackage" for each demo, each with its own docs/ and make.jl or such. I haven't found any examples of this approach though so not sure how to put it all together. Have you seen any examples of a single repo with multiple subpackages in it with partially separate Lit/Doc builds?

(Note to self, I might run into this issue: https://github.com/JuliaDocs/Documenter.jl/issues/319)

mortenpi commented 12 months ago

So the expensive running of the examples are done with Literate, since you pass execute=true generally, right? You could probably invent something where you cache the .md files produced by Literate with GitHub artifacts. But you'd have to add the necessary logic into make.jl and your GitHub workflows.

JeffFessler commented 12 months ago

Cool idea - thanks for the tip! I will look into that. And yes, it is the Literate execute that takes the bulk of the time.