In example, add a multi-choice question with (python) code in the prompt

[mlelarge]

Context: I used mdbook-quiz to make quizzes for a course in python and I struggled a bit with the format of the prompt.prompt for multiple choices question.

What I did: I first thought that you could enter any markdown as for the mdbook. For some reason, this is not the case. As far as I understand, what was causing problem is my poor understanding of the rust parser. In the end, the prompt needs to be only one line and to include some code I did use the following syntax: <pre><code class=\"language-python\"> ... </code></pre> Not sure, this is the right way to do? More examples available at: https://github.com/dataflowr/quiz

A better solution would be that the prompt could be any markdown (i.e. like what is used as input for mdbook) but I do not know if this is possible?

I did not try to make tracing question in python but this might be another nice example to add.

[willcrichton]

You should be able to enter any Markdown. But the default code snippet language is assumed to be Rust. You can change it to python like this:

[[questions]]
type = "ShortAnswer"
prompt.prompt = """
Which Python keyword would go in the blank?

```python
def foo(x):
  if x > 0:
    ____ x;
  else:
    ____ -x;
```
"""

[mlelarge]

Indeed this is working, thanks. But the triple " is crucial, my mistake was to write:

[[questions]]
type = "ShortAnswer"
prompt.prompt = "
Which Python keyword would go in the blank?

```python
def foo(x):
  if x > 0:
    ____ x;
  else:
    ____ -x;

"

This is not working... I do not understand why?
Thank you for your help!

[willcrichton]

That's due to how TOML works. Single-quote is intended for one-line strings, triple-quote for multi-line strings.

See the relevant part of the TOML spec: https://toml.io/en/v1.0.0#string

[mlelarge]

Thanks! Sorry to bother you but I did not manage to put code in the prompt.answer. Here is my example:

[[questions]]
type = "MultipleChoice"
prompt.prompt = "What is the order of the outputs?"
prompt.choices = [
    """
    tensor([0.1576, 0.5853, 0.1312], grad_fn=<SigmoidBackward>)
    tensor([0.0548, 0.9317, 0.0628], grad_fn=<SigmoidBackward>)
    """,
    """
    tensor([0.0548, 0.9317, 0.0628], grad_fn=<SigmoidBackward>)
    tensor([0.1576, 0.5853, 0.1312], grad_fn=<SigmoidBackward>)
    """
]
answer.answer = 0
context = """
As learning goes on, `m(input)` gets closer to the `target`.
"""

I tried

prompt.choices = [
    """
    ```python
    tensor([0.1576, 0.5853, 0.1312], grad_fn=<SigmoidBackward>)
    tensor([0.0548, 0.9317, 0.0628], grad_fn=<SigmoidBackward>)

""",
"""
tensor([0.0548, 0.9317, 0.0628], grad_fn=<SigmoidBackward>)
tensor([0.1576, 0.5853, 0.1312], grad_fn=<SigmoidBackward>)
"""

]

but it did not work.

[willcrichton]

Can you clarify what you mean by it did not work? I would expect the first of your choices to be rendered as code.

[mlelarge]

You are correct. I was expecting the same behavior as for the field prompt.prompt. So I guess my question is how do you mix text and code in prompt.choices. More fundamentally why is it not the same rendering of markdown for prompt.prompt and prompt.choices?

[willcrichton]

Can you post a screenshot of the rendering? That will help me understand why it's not what you expect.

[mlelarge]

Sorry for not being clear. The rendering is fine

but if I am adding some text, it is detected as code:

Really no big deal, you should probably not worry about such details....

[willcrichton]

Oh I see, sorry for the late reply. The issue is that in Markdown, you can write code blocks simply by indenting code. For example:

This is text.

    This is code!

In your TOML file, your strings look like:

[
    """
   the order is:
    ```python
    tensor([0.1576, 0.5853, 0.1312], grad_fn=<SigmoidBackward>)
    tensor([0.0548, 0.9317, 0.0628], grad_fn=<SigmoidBackward>)
    ```
    """,
]

The fact that """ is indented does not cause subsequent lines to start at the same indentation level. Therefore Markdown sees "the order is" as being indented, and assumes it's in a code block. I think this is what's happening, anyway.

The solution is to make sure your multi-line strings are not indented.

[willcrichton]

I'm going to go ahead and close this as resolved.