Placeholders in magic

[dsyer]

The Python kernels seem to support $var style placeholders in magic lines. This kernel seems not to:

var slf4j_version = "2.0.13"

%dependency /add org.slf4j:slf4j-simple:$slf4j_version
%dependency /resolve

Adding dependency org.slf4j:slf4j-simple:$slf4j_version
Solving dependencies
StrictConflictException: org.slf4j#slf4j-simple;2.0.13 (needed by [notebook#jupyter-kernel;version]) conflicts with org.slf4j#slf4j-simple;${slf4j_version} (needed by [notebook#jupyter-kernel;version])
...

so I assume it's a choice that the kernel makes? Does that make sense?

[padreati]

The whole part of the magic parsing and execution is built inside the kernel. When a cell has mixed parts, both java code and magic code, the content is split in groups of lines and are interpreted sequentially, sending each group to the appropriate parser. The two parsers are somehow independent, the java one is built in jshell and the magic one is completely new.

There are two strategies to make them interoperate. The first strategy would be to create a syntax to be able to reference variables from magic parsers. That can work, but it would be limited to variables. The next question would come as why not allow to call a method from an object, with eventual parameters which are also variables? I think it can grow in complexity a lot.

But there is a second way to implement that. Why not expose all the magic commands also in the form of utility java methods? For example to be able to call something like 'Dependency.add(String id)' which will have the same effect as writing the magic line? The parsing will be handled by jshell, which already exists, and you have the freedom to do much more complex stuff like writing loops, conditionals or whatever to create custom logic appropriate for you case.

I strongly incline for the second option. I will try to find a way to have a common uniform syntax to be able to call all magic code like that. What do you think about it?

[dsyer]

I guess I would be happy just to be able to do it, but I prefer the placeholder approach because that's the way that the Python kernel works. JShell exposes variables, so they should be fairly accessible?

[padreati]

The variables from JShell are accessible and with some care they can be transformed in strings, that is not an issue. I am not that confident about the use of $, however. That symbol can appear in a java variable name or in a class name, which is not the case for Python. A better replacement has to be found. In Python you also have ${..} for expressions. That perhaps might be used or something similar. Additionally there is a need of some escape sequence in case you want to use those chars without interpolation.

But the implementation in itself to be complete might require two things: nested expressions and expression interpreter. If the first can be left out, the second seems more natural and is not trivial (in the best case that can be handled by jshell if there is a way, otherwise it would require a lexer, a parser and a small interpreter). Actually the same problem arise similar with string interpolation in Java which recently was left out from preview due to many discussions.

I will think more about it, it would be great to be implemented. In the meantime I will expose some API so that any magic line can be called also from normal Java code.

[dsyer]

I had already spotted the issue with Python style placeholders, but I think ${...} is not hard to implement (and I would choose \\${ as an escape for consistency with other JVM tools). That makes the parsing part quite easy, unless I'm missing something.

The API will be useful anyway probably.

[padreati]

I will implement it using placeholders, just I did not made up my mind on which placeholders to use. Additionally I will add also a way to call magic commands directly in Java code, but that will be subject to other issues.

[padreati]

I have implemented using \{...} since it looks like string templates. A double \\ acts like an escape, no interpolation done.

The interpolation is available for all magic commands. Thus things like the folloing will work:

String cmd = "/add";
String lib = "com.github.javafaker:javafaker:1.0.2";

%dependency \{cmd} \{lib}
%dependency /resolve

It will be released with a new bump version 2.0.0. The reason is that the dependency handling was changed with breaking changes because Ivy was replaced with MiMa (basically Maven).