Make Cesil aware of nullability annotations, and enforce certain correctness invariants implied by that

[kevin-montrose]

Makes Cesil aware of null annotations, as raised in #25.

Visible changes

• Formatter, Getter, InstanceProvider, Parser, Reset, Setter, and ShouldSerialize will now gain one of three "handlings" for the rows they take or produce and values they either take or produce. These handlings, while not visible themselves, can inject null checks into the generated code that reads and writes rows.
  • These "handlings" are inferred based on null annotations injected by the compiler when nullable reference types are enabled, as well as the types of rows and values
    • This sort of unifies .NET's (really C#'s) notion of nullability: both string? and int? are considered nullable in the same way, while string and int are equally non-nullable
    • As a consequence, where Cesil allows for forbidding nulls (detailed more below) that capability applies to both "actually nullable" types (like string?) and "semantically nullable" types (like int?).
  • For cases where rows are optional (as in static Setters and Getters) no "handling" is inferred.
• When code for reading/writing is generated, if the runtime type of a row or value would allow violation of it's associated "handling" either a pre- or post- invocation null check is inserted as well
  • For example, if a Getter returns a non-nullable object an explicit check will be made to ensure that it did not return null (test)
  • It is legal to use different wrappers together if their "handlings" are incompatible (for example, a Setter that takes a non-nullable row with an InstanceProvider than may produce null rows)
• When the semantics of the runtime would not actually allow null values, but the type of a row or value would, no null checks are inserted
  • For example, invoking an instance method will fail fast (a null reference exception will be raised by the runtime) so no null checks are inserted before invoking Getters backed by instance methods
  • Similarly, constructors cannot produce null values so InstanceProviders backed by constructors will not have a null check inserted after they are invoked
• The following types gained AllowNullRows() and ForbidNullRows() methods, which return wrappers that are identical except for their "handling" of rows: Getter, InstanceProvider, Reset, Setter, and ShouldSerialize
  • It is illegal to call these methods on Getters, Resets, Setters, and ShouldSerializes that do not operate on rows
• The following types gained AllowNullValues() and ForbidNullValues() methods, which return wrappers that are identical except for their "handling" of values: Formatter, Getter, Parser, and Setter
• It is illegal to call any of the AllowNullXXX() methods if the relevant type is not "semantically nullable", that is if it is not a reference type or a nullable value type.
• For the wrappers that support chaining via Else (like Formatter, when two wrappers are chained their "handling" will be combined so that runtime checks will still fail early
  • For Formatters this means that if either wrapper forbids nulls, the chained pair will also forbid nulls
  • For other wrappers, if either wrapper allows nulls then the chained pair will also allow nulls
  • Importantly, chained wrappers still only have single null pre-or-post-conditions injected. The "handling" applies to the whole wrapper, not to the original pieces of the wrapper
    • This means, for example, if you have two Parsers one which creates string and one which creates string? and then chain them (creating a new Parser which creates string?) no exception will be raised if, when invoked at runtime, the first Parser produces a null since it is legal for the whole (chained) Parser to produce nulls
  • If a chained wrapper has any AllowNullXXX() or ForbidNullXXX() methods called, the change applies to the wrapper as a whole, in keeping with the previous point.

Commentary

I went back and forth on how much checking to do and when to do it (that is, whether to do it at "compile" time when readers/writers are created or at "runtime" when they are used), as well as how much client's should be allowed to "subvert" null handling.

Ultimately I settled on "doing it all at runtime" and "clients can subvert basically everything". I experimented with failing faster, and imposing more restrictions on clients, but ultimately it was too easy for code that "obviously" works to cause validation to fail.

The end goal is to, by default, fail quickly and helpfully if reading or writing values with subvert the constraints implied by nullable reference types I think this approach achieves that, while also allowing for clients to override or customize Cesil's behavior.

Internal notable changes

• Introduces (and weaves throughout) NullHandling
• Additions to Utils to support the above

[SmithPlatts]

I think technically you can end up with a null this, but practically speaking it's not something that happens in sane code

Inside an object? Yes it's rare but can happen. Inside an extension method? Oh it certainly can and bit me recently...

[kevin-montrose]

I think technically you can end up with a null this, but practically speaking it's not something that happens in sane code

Inside an object? Yes it's rare but can happen.

Yeah, I think it's practically only when you have call instead of callvirt on a non-virtual instance method. The C# compiler will generate callvirts so you get a null check (and then the JIT will de-virtualize them), so you tend to be doing something "weird" to get into this case.

Inside an extension method? Oh it certainly can and bit me recently...

Extension methods are actually static methods, so there isn't actually a this pointer to be weird about.

---

I've made all the appropriate changes to this branch I think, I'll merge into vNext in the next day or so... want to give myself some time to think through any additional gotchas. Expanding tests to cover all of this definitely exposed some additional nuances, which has given me a bit of pause about just merging it.