Can't seem to run the cell with magic `%spark.conf` and `%md`

[flowy0]

Hi, It seems like the magic command %spark.conf is not recognised as cell that can be run. Any plans to support this?

Also found that the cell with %md is also ignored.

[allen-li1231]

Hi, I guess it is your cell language setting mismatches with the interpreter in magic command that causes the code not run as expected. I didn't mention in the readme but the magic command does not actually control how vscode identifies the language of the corresponding cell, you will have to also select the language mode in the bottom right corner of the cell to desired one:

Then try run again with your code.

If this is not the case that you have encountered, is the spark.conf a legitimate imterpreter? Afaik it may refer to a method of spark instead of a kernel that runs code. If you have set the correct language of the cell, didn't solve the problem, and the same code works in the original web notebook, would you like to provide a screenshot on the code cell and the expected output?

[flowy0]

Hi, I guess it is your cell language setting mismatches with the interpreter in magic command that causes the code not run as expected. I didn't mention in the readme but the magic command does not actually control how vscode identifies the language of the corresponding cell, you will have to also select the language in the bottom right corner of the cell to desired one: Then try run again with your code.

If this is not the case that you have encountered, is the spark.conf a legitimate imterpreter? Afaik it may refer to a method of spark instead of a kernel that runs code. If you have set the correct language of the cell, didn't solve the problem, and the same code works in the original web notebook, would you like to provide a screenshot on the code cell and the expected output?

For md, i did login to the Zeppelin UI, the markdown cell is rendered correctly. Is this an expected output when i run a markdown cell?
eg:

For spark.conf:

here is a sample of what i run in the regular zeppelin notebook, this sets some custom configs for the spark kernel. This does not have any output, however, i can set specific spark configs easily.

%spark.conf
spark.hadoop.fs.s3a.access.key {spark.ecs.user}
spark.hadoop.fs.s3a.secret.key {spark.ecs.password}
spark.kubernetes.executor.podNamePrefix g-plots-35p
zeppelin.k8s.interpreter.container.image my-python-interpreter-image
zeppelin.k8s.spark.container.image spark-interpreter image

[allen-li1231]

For md, yes it is expected. One of the differences between Zeppelin UI and VSCode cell is that in VSCode, the magic command is treated as part of the markdown context when redering, while Zeppelin UI automatically hides the magic command when rendering.

For spark.conf, could you try setting the language mode of the cell to any language except for markdown? Despite the code being highlighted weird, can it be run anyway now? From my end:

FYI shortcut to run a cell: ctrl+enter.

The code seems like a shell script. I will add support to shell in the language mode in the next version.

[allen-li1231]

@flowy0 Let me know if you have more questions, otherwise this issue will be closed in 3 days.

[flowy0]

For md, yes it is expected. One of the differences between Zeppelin UI and VSCode cell is that in VSCode, the magic command is treated as part of the markdown context when redering, while Zeppelin UI automatically hides the magic command when rendering.

For spark.conf, could you try setting the language mode of the cell to any language except for markdown? Despite the code being highlighted weird, can it be run anyway now? From my end: FYI shortcut to run a cell: ctrl+enter.

The code seems like a shell script. I will add support to shell in the language mode in the next version.

After I changed it to run as Python, it seems to be fine and take effect. Thanks so much!