[rule] TypedDict upgrade (dicts with fixed keys)

[lmmx]

via

Motivation/rationale:

• Go had/has a popular web tool for changing JSON into a Go spec.
• Handling non-required keys is finicky [ergo you want to automate it]
• Handling non-req. keys is ultimately simplified by TD initialisation constraints (cannot use conditionals in the argument itself, cannot pass kwargs either or mypy will reject it)
• https://github.com/python/mypy/issues/8890
• tweet

TypedDict must be constructed directly in one go, and so strictly prohibits conditional kwargs

i.e.

• MyTD({"a": 0}) if True else MyTD({"a":1, "b": 2}) is OK
• MyTD({"a": 0, **({"b": 2} if True else {})) fails mypy
• MyTD(dict(a=0)) if True else MyTD(dict(a=1, b=2)) is OK
• MyTD(dict(a=0), **({b=2} if True else {})) fails mypy

More generally, would like to be able to populate multiple files from the output (in this case: the source file and append the types to a types module) but this could be done in steps (‘pipelining’ separate rules perhaps)

i.e. {"some_name": 0} could be automatically turned into [presuming the name SomeName doesn’t exist):

SomeName = int

class AutoGenDict1(TypedDict):
    some_name: SomeName

Requirements:

• [ ] Upgrade dicts w/ fixed keys to TypedDict classes
• [ ] Support nested TypedDicts
• [ ] Support non-required keys with subclassing without total=False in 3.8, and the NonRequired flag in 3.11+
  • Not sure if 3.8 has the total param, not in docs but apparently in PEP so hopefully https://stackoverflow.com/a/71601675

[lmmx]

• Business need/value: save time, save mental energy for non-rote tasks (like data model) not syntax transformations
• Data needed to solve this: some simple programs (easy to make) and tricker nonrequired keys with conditionals
• Risks: if it doesn’t work, risks wasting time? but no more so than continuing to do this manually
• Responsibilities must not make development harder by breaking programs (i.e. must preserve parseability)
• Operationalisation: what does a deployed solution look like?
• Go/no go: vast majority of TypedDict candidates could be upgraded for clarity, probably not all will need
• Current business goal: in past 24 hours I’ve spent more than an hour of dev time on something automatable
• Definition of Done: should be able to pass in a module with fixed dicts and upgrade them to TypedDict, with nesting
• Agreed delivery requirements: must retain order in original file, must compile before/after
• Early assumptions: this is a straightforward task, nonreq. keys can be supported in 3.8 equivalently as in 3.11+
• Timeline ?
• Data small simple programs and conditional nonreq. keys (examples above will be sufficient)
• Models: different ‘models’ of the language for different versions, no ML models
• Route to deployment: various, 1. in-vim as ‘fixer’ on whole file via ALE, 2. in-terminal whole file via bash, 3. in-terminal select few lines via arg passed, 4. in-vim through some ALE route to apply to a few lines (unknown if feasible)
• Go/no go for iteration: stop iterating if I have to contribute a language outside my wheelhouse (so this means patching ALE for new functionality would probably be too difficult)
• Additional resources: can lean on David Porter’s talk at Typing Summit, typing gitter, Python IRC, but should be straightforward enough to be self-contained