Literate programming versus REPL-based tutorials

[non]

I feel like I've heard people ask whether Tut can support literate programming in Scala. And when I try to explain my (perennially-unreleased) sbt-plugin for literate programming (https://github.com/non/literati), people always say "Oh, like Tut right?"

It might be nice for me to try to fold my work into Tut (which people have heard about and seems to be getting some momentum) rather than having a totally separate plugin for something that seems very similar (at least on the surface).

The main difference between the two comes down to how the compiler is used.

Tut (AFAIK) wants to paste code into one (or more) REPL sessions to ensure that when a reader opens a REPL, the code examples all work as advertised (at least, with the right classpath). It also wants to (potentially) rewrite your Markdown to show the REPL output you'd expect to see.

Literati wants to transform your markdown file into a Scala file (whose line numbers match), and then run the standard compiler on that, producing bytecode, jars, and so on via _managedsrc. This ends up looking a lot like codegen. It doesn't transform the Markdown in any way, since the assumption is that the code is compiled, not run.

I could imagine using a Markdown header, like Jekyll does, to "configure" Tut's behavior. This would be the current (default) behavior:

---
mode: repl

---

and this might be a configuration for literate programming:

---
mode: compile
package: my.example.project

---

(Other optional settings might include object: TopLevelObject or app: TopLevelApp, to optionally wrap the code in an object (or an object extending App).)

The nice thing is that we have one tool to do Markdown-related Scala programming. The downside is that these two modes will share almost no code -- their strategies are orthogonal.

What do you think?

[non]

One other argument against is that you'd probably want to put the "tut" files and the "literate" files in two different locations, to be clear on their different semantics. This might also mean that you don't need a mode parameter.

[tpolecat]

I like the idea of one-stop shopping for markdown-centric Scala programming. I wonder if it would be useful to have tut-style embedding in the literati source code, so your end result (after two passes) would be an annotated source document that includes tut-expanded examples. Dunno.

My dream artifact is an integrated reference with annotated source, inline examples, cross-referenced laws/tests, tutorials, text and type-based search, typeclass-aware nav/indexing … but that's still a ways off I guess :wink:. No harm in dreaming.

In any case I would love to combine these projects even if it doesn't make immediate technical sense. It seems like it could grow into something more integrated. I'm planning a total rewrite (not a big deal, it's like 200 loc) for 0.5 so maybe it would make sense to see if we can fold stuff together in that timeframe (end of the year-ish?). What do you think?

[non]

@tpolecat If we could do a good job of signaling the difference between the tut-expanded examples and the actual source code, I think integrating both together sounds pretty rad.

Your dream is my dream.

Let's do it! That time frame sounds OK to me. One nice thing about things being orthogonal is that I can just try to get my stuff working correctly, and then paste it into whatever you end up with.

[adampingel]

I read this thread with great interest two weeks ago. Nothing new to add other than enthusiastic support. My need is more for the literati-style documentation, but I'd love to see that integrated with Tut.

[adelbertc]

:+1:

The flow I wanted was something similar to this:

Write a Markdown file, perhaps in the style of Remark.JS, or just plain Markdown. From there I can use existing Tut functionality to generate slides, or just make sure my code examples compile and use it for a "live" coding session. For the "live coding" use case, where others would upload slides, I'd like to be able to point people at the source file with irrelevant parts perhaps commented out (like literati I believe) so folks can copy/paste the entire file into a REPL session or some source file.

You would then be able to go from a single Markdown file to something you use to present, as well as something you could give people to dump into the REPL and play around with.

[aryairani]

Some related links:

dotlit:

• https://github.com/zoopdoop/dotlit/blob/master/README.lit.md
• https://github.com/daithiocrualaoich/dotlit-scala (example)

scalocco:

• http://grison.me/scalocco/
• http://grison.me/scalocco-src/scalocco.scala.html (example)

I would love to be able to have software requirements (markdown-ish) and implementation (scala) in the same file, probably interleaved; and be able to generate both documentation artifacts with rendered markdown/source, and compiled class files.

Some more random thoughts:

• Does documentation nested within objects or code blocks need to be treated/rendered specially? My impression of literate Haskell is that most things are defined at the top level of a file, but Scala lets us have a lot of nested namespaces.
• What do you do if the order you want to discuss things is different from the order you need to declare them in Scala? Would there need to be some sort of specs2-like splicing mechanism: defining a chunk of code in one place but using it (rendering it) in another place or multiple places?
• IDE (read: IntelliJ) support is super valuable in my opinion. Is there a way to avoid having to choose between docs and interactive support? I guess having all the markup inside of scala comments would be one way to salvage scala IDE support.

[tpolecat]

Thanks for all of this. Added to the queue. :+1:

[laughedelic]

I just want to mention a related project, that we use at work: literator. I didn't like the idea of code generation, so we did it another way round: you write normal scala code and comment it in markdown, the tool generates markdown files which work as documentation, get rendered on github or can be passed further to some automated site-generator.

[aryairani]

@laughedelic literator looks good!

[laughedelic]

@refried thanks :blush: