Generation: allow duplicate _invalid_ input files

[thorehusfeldt]

The following would be useful and shouldn’t lead to a FATAL error:

invalid_answers:
  - '':
      in: 3 7
      ans: ten
  - '':
      in: 3 7
      ans: 11

[mzuenni]

The reason for this is that the hash of testcase.in is used as a path... and then writing different .ans files would break stuff. However, if the '.ans' is provided we could just include it into the hash?

(this would still break if the .ans is generated in some other way)

[thorehusfeldt]

In particular, the contents of

• https://github.com/RagnarGrootKoerkamp/BAPCtools/blob/master/test/problems/identity/data/sample/3.in
• https://github.com/RagnarGrootKoerkamp/BAPCtools/blob/master/test/problems/identity/data/invalid_outputs/4.in
• https://github.com/RagnarGrootKoerkamp/BAPCtools/blob/master/test/problems/identity/data/invalid_outputs/6.in

are all

1

Since I’m now input-validating the data/invalid_answers (which is the new place for invalid_outputs), the test suite, in particular bt validate --input --problem identity breaks the pytest. (Validation returns 1.)

[mzuenni]

Hm, in generell we could rework the caching into multiple directories one for inputs and one for answers, that should fix this issue?

[RagnarGrootKoerkamp]

I believe this was fixed with https://github.com/RagnarGrootKoerkamp/BAPCtools/pull/333/commits/5d1a59eb10148639425e3d262cedac34f8435c75 as part of #333 right?

We'll probably need to extend that for data/invalid_output, where both in and ans can be the same but only the (to be added) out: key may differ.

[thorehusfeldt]

Feel free to implemented the extension suggested above; out: keys are now a thing. The caching is opaque to me, so I don’t want to touch it.

[mzuenni]

@RagnarGrootKoerkamp i added this, can you take a look?

[thorehusfeldt]

I tried to generator-fy the testcases in invalid_{inputs, answers} in test/identity like so:

  invalid_inputs:
    data:
      "2":
        in: "-1"
      "3":
        in: "1001"
      "7": 
        in: "1\n\n"
      "8":
        in: " 0"
      "9":
        in: "\n0"
  invalid_answers:
    data:
      "4":
        in: "2"
        ans: "-2"
      "5":
        in: "199"
        ans: " 199"
      "6":
        in: "3"
        ans: ""

But I get

NameError: name 'ans_hash' is not defined

and am a bit out of my depth. Maybe you could add the above, and try to make it work with identical input files, so we could instead have

invalid_answers:
  data:
    -'':
      in: "1"
      ans: " 1"
  -'':
      in: "1"
      ans: "1 "

and so on. Once that works, we should (finally) reach the Holy Grail of this entire quest and add

 invalid_outputs:
   data:
    -'':
        in: "1"
        ans: "1"
        out: "2"

I’d be grateful if somebody did this to identity.

[thorehusfeldt]

I think this is solved by feat/invalidation.