Make auto-numbering of `bt generate` consistent with DOMjudge numbering

[mpsijm]

We should auto-number samples so that DOMjudge's internal numbering is the same as ours (so if there are two samples, 3.in would be the first secret case).

When fixing this, please double-check that samples with leading zeroes are correctly picked up by the PDF. We've had some instances where we a file like 001.interaction is not picked up by the PDF.

[mzuenni]

Do we want to zero-pad the samples like the secret cases or it is sufficient to have secret cases start at |samples|+1? And what is the expected behaviour if there are other folders in /data?

[thorehusfeldt]

For reference: Summer 2023 discussion of test case numbering by bt generate.

[thorehusfeldt]

The only way I can see this work with the current decisions about automatic numbering is to introduce a setting for it, maybe as part of #testgroup/#config in the schema:

data:
  secret:
    numbering:
      start: 3
      padding: False
    data:
        foo: my_generator 344

We’d probably want to invent reserved values for start like start: sibling.

This seems to require a lot of thought to do well. (An altenative is to have a single rule that kicks in exactly if the directory structure is data/secret/list_of_cases and data/sample/list_of_cases and nothing else.

[RagnarGrootKoerkamp]

@thorehusfeldt Currently auto-numbering is implemented as follows:

1. Collect all auto-numbered cases in generators.yaml.
2. Sort them lexicographically.
3. Number them 1 to n.
4. Left pad all of them with zeros.

So this already works for samples. I think all that's needed is to change the template from

samples:
  data:
    1: ...
    2: ...

to

samples:
  data:
    - '': ...
    - '': ...

This will left pad with zeros, but I think that's OK? (Or would that leak information to contestants who can see their filenames in the UI? I would hope those are not leaked but am not sure.)

[thorehusfeldt]

You’re right—the default setting already provides the solicited functionality!

Maybe this is a non-issue.

[mzuenni]

I want to note a few thinks

1. This leaks (some) information as the sample.zip contains the file names?
2. The latex code does not currently find 001.in (but that's easy to fix)
3. If we decide to ignore 1. and fix 2. we might want to change the example generator.yaml?

[RagnarGrootKoerkamp]

We could also hardcode that samples specifically not padded? That sounds simpler.

1. Either way samples.zip probably should rename cases to be (zero-padded?) 1 to n.
2. Hmm fixing the latex does sound a bit annoying though.
3. Yes we should update the generators.yaml example and/or templates.

[mzuenni]

1. ok, samples are already renamed (without padding)
2. and 3. i will take a look

[mzuenni]

It seems like there is not much to do here:

• The generators.yaml in skel already looks good to me.
• We can change the test problems (?)
• We can change the generators.yaml in doc (?)
• The bapc way of including samples already works (only the latex code used at GCPC and NWERC may break)

[RagnarGrootKoerkamp]

Thanks for checking :)

I'd say test problems can stay as they are, but updating doc/generators.yaml sgtm.

[mzuenni]

I think changing the generator.yaml in doc is not that easy... it mainly explains what can be done, and right now automatic numbering is the last point but samples are the first point. Therfore, I am not sure what we want here...

[RagnarGrootKoerkamp]

ah fair enough; then nothing left here I suppose, apart from making sure we actually do this next season.