Numbering of justification is numeric instead of alpha

[mhamburg]

In the sample exam, the input md file contains answers and justifications which are numbered 1-4 (or more).

The pdf generator is required to convert the numbering (1-4 or more) to lower-case alphabetic characters (a-d or more).

This used to work before. Currently, it only works for the questions.pdf. Coversion is not done for the answers.pdf.

Example screenshots:

- Input:

- Correct output of June 18th:

- Incorrect output today:

[danopolan]

@Witiko can you have a look please? It's TA repo which is affected for sure

[Witiko]

@mhamburg Currently, only the section ## answers is parsed and the answer numbers converted to alphanumerical letters. In other sections, you would still manually write alphanumeric letters.

For example, in the document from your screenshot, you still write correct: d, not correct: 4. Similarly, the section ## justification is not parsed, since it may contain any free-form markdown text, not just a list of answers. To receive the output from your screenshot with the current implementation, you would write:

## justification
a) Is not correct. In incremental models, the test scope is focused on but not restricted to new and changed features. The TA must also pay attention to regression tests.  
b) Is not correct. Each increment usually requires not only test design, but also test analysis, test implementation and test execution.  
c) Is not correct. For each increment, the TA work may be organized differently. Therefore, the time of involvement of the TA may vary in various increments.  
d) Is correct. Developing each increment usually requires to perform the same set of activities within a test process for that increment. Therefore, the TA performs similar tasks for each increment.  

If section ## justification will always contain a list of answers, then I can update the implementation, so that it processes both sections ## answers and ## justification the same way. Furthermore, we can also change correct: d to correct: 4 for consistency. Then, the use of lower-case alphabetic characters would be just an implementation detail of the rendering that could be changed at any point and not be part of the question definitions. What do you think?

[mhamburg]

We must discuss this in more detail, because the section ##justification does not always contain a simple list of answers. Moreover, it may contain other enumerations that must not be labelled with letters.

[mhamburg]

I discussed with Daniel that numbering should only be converted to lower-case letters if it starts right after the heading ##justification.

[danopolan]

So, let's sum it up:

• Justifications could be a list (numbered or latin lettered) or a free text with possible usage of list within this text (as done here)
• usage of letters in ##justification section is giving us a weird output in the PDF
• correct answer is listed as a letter not number

So I would like to:

1. Have numbered lists used in both ##answers and ##justification sections
2. In case ##justification has not only a numbered list, but e.g. also some free text as well, handle it like a free text
3. The correct answer is indicated as a number, not a letter
4. Text "Is correct", "Is not correct" could be added automatically to all justifications in case of a numbered list, so authors has less text to input.
5. Have the replacement of numbers to letters for correct answer, answers, justification and addition of "Is correct", "Is not correct" done for all formats including DOCX

@Witiko would this be possible?

[Witiko]

usage of letters in ##justification section is giving us a weird output in the PDF: [image]

What is weird about the output? Is the issue that the output is "a." and not "a)" as in the June 18 screenshot in https://github.com/istqborg/istqb_product_base/issues/113#issue-2547782443? That should be solved by writing "a)" rather than "a." in the document questions.md.

@Witiko would this be possible?

Of course. Is Monday soon enough?

[danopolan]

That should be solved by writing "a)" rather than "a."

Oh sure. I did not realized that.

Of course. Is Monday soon enough?

Sure. No hurry. We have few weeks.

[Witiko]

I was looking at our example questions and here are some issues that I see with the approach outlined in https://github.com/istqborg/istqb_product_base/issues/113#issuecomment-2375177804 and in https://github.com/istqborg/istqb_product_base/issues/113#issuecomment-2375280499:

For some questions, there is a significant potential for incorrectly guessing whether the text in section ## justification is a list of answers or a different list, even when there is no free text:

# metadata
lo: 1.4.2
k-level: K2
points: 1
correct: b

## question
Which of the following factors (1–5) have SIGNIFICANT influence on the test process?

1. The SDLC
2. The number of defects detected in previous projects
3. The identified product risks
4. New regulatory requirements forcing
5. The number of certified testers in the organization

## answers
1. 1, 2 have significant influence; 3, 4, 5 have not
2. 1, 3, 4 have significant influence; 2, 5 have not
3. 2, 4, 5 have significant influence; 1, 3 have not
4. 3, 5 have significant influence; 1, 2, 4 have not

## justification
1. Is true. The SDLC has an influence on the test process
2. Is false. The number of defects detected in previous projects may have
   some influence, but this is not as significant as i, iii and iv
3. Is true. The identified product risks are one of the most important
   factors influencing the test process
4. Is true. Regulatory requirements are important factors influencing the
   test process
5. Is false. The test environment should be a copy of the production
   environment but has no significant influence on the test process

Here, the list in section ## justification would have been detected as a list of answers and displayed with letters a–e. However, this would be incorrect, since the list items refer to the factors 1–5, not the answers.

For other questions, there is free text that contains a list of answers that would still need to be hand-written as a), b), c), d) instead of 1., 2., 3., and 4.:

# metadata
lo: 1.4.2
k-level: K2
points: 1
correct: a

## question
Given the following test activities and tasks:

A. Test design
B. Test implementation
C. Test execution
D. Test completion

1. Entering change requests for open defect reports
2. Identifying test data to support the test cases
3. Prioritizing test procedures and creating test data
4. Analyzing discrepancies to determine their cause

Which of the following BEST matches the activities with the tasks?

## answers
1. A-2, B-3, C-4, D-1
2. A-2, B-1, C-3, D-4
3. A-3, B-2, C-4, D-1
4. A-3, B-2, C-1, D-4

## justification
The correct pairing of test activities and tasks is:

A. Test design – (2) Identifying test data to support the test cases
B. Test implementation – (3) Prioritizing test procedures and creating test data
C. Test execution – (4) Analyzing discrepancies to determine their cause
D. Test completion – (1) Entering change requests for open defect reports

Thus:

a) Is correct
b) Is not correct
c) Is not correct
d) Is not correct

This seems inconsistent and confusing to authors: Why do I write 1. here but a) for another question? Why is "Is correct" and "Is not correct" automatically added in the output if I write 1. but not if I write a)?

Furthermore, both example questions show the author using differently-numbered or differently-lettered lists for factors, activities, and tasks. If it's unclear from the markdown text that the answer numbers will be replaced with lowercase letters, then the author might also use lowercase letters for their other lists, resulting in confusing output where multiple lists use lowercase letters and references to list items are ambiguous.

For these reasons, I would suggest keeping the automatic replacements to a minimum and use letters for answers everywhere, including section ## answers. This will keep the output more predictable and consistent for authors, which seems more important that the marginal time saved by not typing "Is correct" and "Is not correct" manually.

Specifically, here is my counterproposal to the solution from https://github.com/istqborg/istqb_product_base/issues/113#issuecomment-2375280499:

1. Use letters a), b), c), and d) everywhere: in field correct: and in sections ## answers and ## justification.
2. In both sections, display both a. and a) as a) to ensure consistent output, as discussed in #113 (comment).

Ad 1.: How many answers can there potentially be? More than five?

[danopolan]

I was looking at our example questions and here are some issues

Good catch @Witiko. I did not realize there are Sample Exams which are more complicated since they are converted from existing documents, not newly created documents in MD.

Specifically, here is my counterproposal to the solution

The main motivation for using 1-4 instead of a is that Markdown supports numbered lists and is much easier to work with then. Meaning newlines are assured without double spaces at the end of a previous line, indents are added, etc. This helps authors while working in GitHub or VS code a lot. Also formatting as list in DOCX output is assured. And I am not sure there is a way around it with a-d

Ad 1.: How many answers can there potentially be? More than five?

It's 1 of 4 or 2 of 5. So, the maximum of 5 answer options with 2 correct ones.

[Witiko]

Also formatting as list in DOCX output is assured. And I am not sure there is a way around it with a-d.

We can maintain line breaks when we convert questions.md to DOCX, so that instead of a) foo b) bar c) baz, the DOCX output would contain something like this:

a) foo b) bar c) baz

For all intents and purposes, this seems indistinguishable from an actual list.

Meaning newlines are assured without double spaces at the end of a previous line, indents are added, etc.

In sample exam questions, a list item with an answer / justification would typically be one line of code such as this:

a) Entering change requests for open defect reports

Even if the author wanted to wrap a longer answer / justification across several lines, they are in a non-nested list, so no indent is necessary on continuation lines:

a) Entering change requests for
open defect reports <!-- this is OK -->

Furthermore, even if the text editor added a double space at the end of a line, a double space at the end of a list item does not affect the output:

a) Entering change requests for open defect reports␣␣
                                         <!-- this ^^ is also OK -->

In short, whereas I see the value in text editors that can provide help with formatting Markdown files, I don't think they make much difference in formatting a list of answers / justifications specifically.

If you insist to use numbered lists for answers / justifications, then I don't have a good solution for the issues that I described in https://github.com/istqborg/istqb_product_base/issues/113#issuecomment-2383125226.

[danopolan]

Ok, let's go with lettered list like a-e in all occurrences in questions.md. So the requirements would be:

1. Have numbered lists used in ##answers, ##justificationand correct: sections
2. inputs a. or a) are both valid
3. All output formats have correct formatting with lettered lists