Discussion: REPL design thoughts

[paulbutcher]

Magnus asked me to put together some thoughts on what I would expect from (and just as importantly, what I would not expect from) a Flix REPL. To my mind, there are two key use cases that a Flix REPL should enable:

1. Learning and Exploration: learning about, and exploring the idiosyncrasies of, the Flix language and libraries.
2. Debugging and Visibility: examining the behaviour of an existing Flix program to understand why it’s behaving in the way that it is.

What is definitely out of scope, I think, is:

• Modifying an existing program: Some languages allow an existing program to be modified “on the fly” (redefining functions, data structures, etc.). I don’t believe that a Flix REPL should attempt to provide this.

Possible User Experience

Note: What follows are concrete examples of one possible approach to stimulate discussion. Everything is up for discussion!

User starts the REPL within an existing Flix project:

% flix --repl
Welcome to Flix!
… introductory help text …
User> 

The User> prompt indicates that they’re in the default User namespace.

Note: We might choose to support running the Flix REPL without an existing Flix project, but I don’t think that that’s high priority.

Note: We might choose to allow users to define their own prelude for the User namespace (for example within ~/.flix/ ) in which they can define common helper functions they find useful during REPL sessions.

User explores the Flix language, e.g., trying out a few simple expressions:

User> 1 + 1
2                                                           // tmp1
User> 1 :: 2 :: 3 :: Nil |> List.head
Some(1)                                                     // tmp2
User>
tmp2 |> Option.getWithDefault(0)
1                                                           // tmp3

Note: there are all kinds of different naming schemes we could use to allow the user to refer back to intermediate results, and ways to indicate them visually (e.g. different colours etc.). Some REPLs use positional references (e.g., $1 to refer to the most recent result, $2 the previous one, etc.). My preference would definitely be for named intermediate results.

User explores defining functions, enums, etc…

User> def myFunc(x: Int32): Int32 =
 ...>   x + 1
<function: myFunc>
User> myFunc(3)
4                                                           // tmp4
User> enum Piece with Order {
 ...>   case King, Queen, Rook, Bishop, Knight
 ...> }
-- Instance Error --------------------------------------------------

>> Missing super class instance 'Eq' for type 'Piece'.

>> The class 'Order' extends the class 'Eq'.
>> If you provide an instance for 'Order' you must also provide an instance for 'Eq'.

enum Piece with Order {
                ^^^^^
                missing super class instance
Tip: Add an instance of 'Eq' for 'Piece'.
User> enum Piece with Eq, Order {
 ...>   case King, Queen, Rook, Bishop, Knight
 ...> }
<type: Piece>
User>

Note: the REPL will need to be able to parse “enough” of the input to know whether or not it should display the ...> continuation prompt.

More exploration:

User> let x = ref 1
<ref: Int32>
User> x := deref x + 1
<ref: Int32>
User> deref x
2
User>

(Most of) normal Flix semantics and rules hold. So, for example, redefining a name that already exists is an error:

User> let x = 1
<var: x>
User> let x = 2
-- Redundancy Error --------------------------------------------------

>> Shadowed variable 'x'.

  let x = 2;
      ^
      shadowing variable.

The shadowed variable was declared here:

  let x = 1;
      ^
      shadowed variable.

The obvious exception to the above is that Flix’s rules about unused variables will have to be disabled within the REPL.

The user can refer to anything defined within the project (e.g. calling functions, using types, etc):

User> MyProgramNamespace/Thingy.doSomething(1, “foo”)
... output ...

It should also be possible to run the program within the background, and then examine its state as its running. For example imagine that our program is a web service (not possible yet, but hopefully soon!):

User> :run
Running main...
User> Request/getRequestCount
0

Then later, after the server has serviced some requests:

User> Request/getRequestCount
3

Note: the VSStudio plugin should support a window containing the REPL, and provide a “run and open a REPL on the running program” option.

Once we’ve debugged our problem (whatever it is) we edit the source code of the project and rerun.

Note: no changing the running program in place.

Note: it will be possible to change the state of the running program by calling side-effecting functions.

Note: we'll need to consider side-effects in the implementation, which disallows a "collect everything the user's typed at the User> prompt into a function and rerun every time" approach.

[JonathanStarup]

Looks good! I have often seen repl in situations where people showcase what types things are so I was wondering what your opinion is about displaying types. In the above its only shown here

User> let x = ref 1
<ref: Int32>
User> x := deref x + 1
<ref: Int32>            (note: this should be unit I presume)
User> deref x
2
User>

Should the type display be related to let-bindings, displayed with all values, or shown based on a special repl method?

[paulbutcher]

(note: this should be unit I presume)

Oops, yes, definitely.

Should the type display be related to let-bindings, displayed with all values, or shown based on a special repl method?

That, I'm not sure about. It should be "whatever is most helpful", but I'm not sure what that means, exactly.

[paulbutcher]

How's this as a starting point for when types should be displayed?

• Any value that implements ToString is displayed as-is.
• For everything else, we display its type

So:

User> 1 + 2
3
User> def myFunc(x: Int32, y: String): String = ???
$ Int32 -> (String -> String)
User> enum Piece {case King, Queen, Rook, Bishop, Knight}
$ Piece
User> King
$ Piece

(the intention of the $ (not sure whether this is the best choice, but) being to indicate that what you're seeing isn't the value itself).

[stephentetley]

Note: the REPL will need to be able to parse “enough” of the input to know whether or not it should display the ...> continuation prompt.

GHCi uses explicit brackets for multiline input. Following this would be simpler for the parser.

[paulbutcher]

GHCi uses explicit brackets for multiline input. Following this would be simpler for the parser.

I guess another option would be some kind of explicit "line continuation" character (e.g. a \ at the end of the line)? Not ideal, but not a disaster either.

[JonathanStarup]

You could also, before giving the text to the actual parser, let the user input lines until parentheses match using a simpler parser. This would let you input tuples/records/def using multiple lines (def f(x): Int32 = {)

[magnus-madsen]

GHCi uses explicit brackets for multiline input. Following this would be simpler for the parser.

I guess another option would be some kind of explicit "line continuation" character (e.g. a \ at the end of the line)? Not ideal, but not a disaster either.

I like this as the starting point. Its simple and easy to explain. Later we can go for something more fancy.

[magnus-madsen]

@stephentetley What's the bracket thing? Is it like if you write { then GHCI knows to look for the closing bracket?

(I guess we can not use brackets because we have them in the language, so we would need something else).

[stephentetley]

GHCi uses :{ and :} or it has a multiline mode which looks like it uses a parsing strategy so see if input is finished:

https://downloads.haskell.org/~ghc/8.8.3/docs/html/users_guide/ghci.html#multiline-input

[magnus-madsen]

I would propose the following:

If a line ends with \ then we ask for more input until the line does not.

If you enter \\ on a line by itself then the parser keeps asking for input until you enter \\ on a line by itself.

[magnus-madsen]

Thus:

flix > 1 + \
.... > 2

works and

flix > \\
.... > 1
.... > +
.... > 2
.... > \\

works.

[JonathanStarup]

Another thought I had, the repl can be used instead of the documentation sometimes ("What list is n in using List.split(n)?") by just running the function. Python enhances this further by allowing print(str.__doc__) to see the documentation. I think documentation access would be nice to have in some form.

[magnus-madsen]

@stephentetley I seem to remember you were also a fan of REPLs :) I'd also appreciate your input:

• Are there any use cases or important features we have missed?
• Does Haskell/GHCI allow redefinition of declarations, types, etc?
• What's your current workflow today? Do you use VSCode? The repl? Your own custom thing?

[stephentetley]

It would be nice to run Datalog queries directly (or near directly) in the REPL.

It looks like GHCi allows redefinitions but the new declaration shadows the old one:

https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html#type-class-and-other-declarations

I haven't used the new REPL yet. I use VSCode but I seem to be missing the features more powerful than syntax highlighting. My .vscode doesn't seem to have a flix.jar, but I run with an updated one that tries to track flix/flix head.

[paulbutcher]

I use VSCode but I seem to be missing the features more powerful than syntax highlighting. My .vscode doesn't seem to have a flix.jar, but I run with an updated one that tries to track flix/flix head.

If you place an up to date flix.jar in the root of your project, the VSCode plugin should use that. You should see something like this in the "Flix Compiler" channel of the "Output" window (it should be displayed automatically when you open a Flix file):

LSP listening on: 'localhost/127.0.0.1:8888'.

Flix 0.28.0 Ready! (Extension: 0.75.0) (Using /Users/paulbutcher/Projects/flix-playground/flix.jar)

Do you see that? And if not, what do you see?

[paulbutcher]

It looks like GHCi allows redefinitions but the new declaration shadows the old one:

Do you use this "redefine but shadow" functionality? It sounds to me that it adds quite a lot of complexity (both in terms of the code, and also in the user's mental model) which we should only contemplate if there's a strong reason to do so?

[stephentetley]

Do you see that? And if not, what do you see?

Thanks @paulbutcher - yes this works.

It would be nice to have a setting for compiler path in the Flix VSCode extension then I don't need a copy of the jar in each of my projects.

[stephentetley]

Do you use this "redefine but shadow" functionality?

No, my own GHCi "workflow" is quite primitive - I have a Haskell source file where I write any declarations or function definitions and I load and reload that. At the prompt I almost always just invoke functions I've written in my "doodle" file.

[paulbutcher]

Thanks @paulbutcher - yes this works.

Good to hear. What happens when you don't have flix.jar in the root?

It would be nice to have a setting for compiler path in the Flix VSCode extension then I don't need a copy of the jar in each of my projects.

Understood. One of the things I'm planning on looking at soon, I hope, is packaging, which will consider all of:

• How to determine which version of Flix to use
• How to determine which Java libraries to use
• How to determine which Flix libraries to use

But I want to get the Java interop and REPL workflow working first 😜

[stephentetley]

Good to hear. What happens when you don't have flix.jar in the root?

I think it should delegate to one in .vscode user configuration. For some reason VSCode doesn't download a flix.jar for me on Windows (I've seen it does on Linux).

How to determine which Java libraries to use How to determine which Flix libraries to use

A simple scheme would be to follow Flix package manager which is working very well for me. Look for libraries - jars and Flix .pkg files - in a lib folder at the top level of the working directory.