Render Squiggle code to notebooks

[berekuk]

This is an early draft of an idea that we discussed, that I want to write down in some form.

What are we missing compared to Jupyter-style notebooks to render Squiggle to the nice list of interspersed values and markdown notes?

We already have:

• markdown strings
• docstring comments
• ability to render nested arrays and objects

Example emulating a notebook in Squiggle 0.8.4: https://www.squiggle-language.com/playground#code=eNqrVkpJTUsszSlxzk9JVbJSqlCwVTCNyasEUkYKJfkgdkxedEyegkKMkkdqUap6sUKFVYySDkikQgci7piXopABkauEyVXG5MUq1QIAV4ocbg%3D%3D

Still, it doesn't feel as nice as Jupyter or Observable notebooks.

Key reasons:

• Array output is noisy (includes indices)
• Writing strings in end result means that the strings are not in the right place in the source code
• You have to duplicate variable names in that array
• You could write strings in the source code instead and render to bindings (variables), but that means that you'd have to pick names for each comment

Also, sometimes you might want to render a plot and in 90%+ cases you won't be interested in storing the plot object to the variable. In that case, coming up with a variable name is also annoying.

This makes me think that in addition to variables and result, we need to add a third mode; something generate a notebook output (array of interspersed values) from the code. Some of those values won't be bound to any variables, but the reducer would collect all values and all bindings in the right order and return those as a "notebook" object.

Previous related discussions: #977, #720.

Previously we discussed ideas such as value -> ignore or call keyword. call (or maybe render; see my https://github.com/quantified-uncertainty/squiggle/issues/977#issuecomment-1266901523) is still possible, but for markdown specifically we'd either need some syntax with braces, or multiline strings.

Syntax ideas:

// Option A:
// "render" is a builtin keyword.
render Plot.distFn({ ... })

// Markdown?
render "# header"

render `
# header
lorem ipsum
`

// Or maybe treating strings as markdown is a mistake;
// see https://github.com/quantified-uncertainty/squiggle/issues/2241#issuecomment-1701926918
render markdown(`# header
lorem ipsum
`)

// Option B:
// Rendering markdown is such a popular use case that syntax from (A) still feels too verbose.
// How about this?
markdown `
# header
lorem ipsum
`

// Option C:
// Having too many top-level keywords that affect the parser feels wrong.
// What if we need more than `render` and `markdown` in the future?
// Maybe we need a common prefix for "directives" that are not bindings?
// Examples of other directives:
// - `@inspect` (which is currently a function, but it doesn't have to be)
// - `@env`, to locally change `sampleCount`?
// - something for debugger?
@render Plot.distFn({ ... })
@md `
# header
lorem ipsum
`

// Option D:
// In option C, "@md `" is repetitive: if it's a markdown block, why do you need quotes?
// Also, you might want to use backticks in markdown content, and that would break the parser.
// Is there some other way to quote markdown safely?
// Two possible ideas...

// Option D1:
// In this case, parser would be aware of `@md` directive and switch to markdown-parsing mode.
// Braces are still necessary to mark the end of markdown string.
@md(
# header
lorem ipsum
)

// Option D2:
// Previous example is still fragile.
// Real markdown is robust in the face of formatting issues (any text is a valid markdown),
// but D1 requires parens to be balanced.
// Maybe heredoc syntax would be helpful? https://en.wikipedia.org/wiki/Here_document
@md <<END
# header
lorem ipsum
END

[OAGr]

Quick comments:

• I see this functionality more as "literate programming" than what I think most people think of as "notebooks", unless we add a UI on top.
• I like that this doesn't add too much functionality. Maybe we could consider it "experimental" for a while?
• It's not clear to me how useful this would be.
• I'd guess that it might not be that much work to try out.

[OAGr]

Another option might be to call it, @debug() or @logger() or similar. (For rendering a value to the sidebar)

[OAGr]

A similar way of doing this would be to have "log" statements that rendered to the view. No extra syntax needed. Just, log(normal(5,2)).

This would be really useful for debugging.

[OAGr]

Having some Console.logRunningTime(str) would also be useful, for debugging how much time steps take.

[berekuk]

log(normal(5,2))

This is problematic syntax-wise; usual Squiggle statements are assignments, but this one isn't (previously: call keyword, void type, etc.)

Even if we remove top-level end expressions, we still have end expression in blocks.

[OAGr]

We have inspect, I was imagining that log() could work like that. It could return the variable.

I guess it could be a norm to do:

_ = logger("foo", var10)

[berekuk]

_ = logger("foo", var10)

Two reasons I don't like this approach:

1. _ = ... relies on variable shadowing for _ (assuming it's a variable and not special syntax), and I hope we'll remove variable shadowing eventually
2. logger has a side-effect, so it's not pure anymore, but the interpreter won't be aware of that; this means that we won't be able to do optimizations such as parallel evaluation (which I expect to become very important later on, and I don't want to create a precedent here)