Add "Checker MUST handle broken utf-8 gracefuly" Tenet

[domenukk]

Our checkers must not crash if teams insert random broken utf-8 characters into other team's publicly readable databases. This would lead to an unfair and unpatchable disadvantage to these teams. We should add this to our Tenets and probably add checks for it. For service authors, they should either handle bytes encoded, or handle each decode and decide if they should continue on errors, there.

[Trolldemorted]

Should we generalize it to "MUST gracefully handle all valid output of the service" or something similar?

If my service only accepts uftf8-input the checker should report mumble if it receives gibberish, and this is not utf8-specific but should also be applied to corrupted protobuf messages or whatever.

[domenukk]

True, it should handle all input, however I would explicitly note broken utf-8. For example, broken protobuf would probably (rightfully) result in a mumble service (broken response), while broken unicode can be the result of a fully-functioning service, and should not mumble.

/edit: Checkerfuzzer ETA When

[Trolldemorted]

For example, broken protobuf would probably (rightfully) result in a mumble service (broken response)

It depends, if the checker parses protobuf supplied by users then it can rightfully be malformed, under the assumption that the service does not enforce protobuf validity.

I agree that uft8 is the most common candidate, what about generalizing it and using utf8 as an example?

[domenukk]

How about A Checker MUST gracefully handle all valid output of the service. For example, be careful when decoding utf-8 from user-supplied fields, as they could contain illegal characters.

[domenukk]

This has been addressed, I think?

[ldruschk]

This has been addressed, I think?

Where? It's still missing from the tenets I think

[Trolldemorted]

What do you think of when decoding utf-8 from user-supplied byte array fields?

If a service is expected to return a valid string the checker should say mumble after all