Serialiasation support for generic types would allow for serialisation of more complex nested classes/dataclasses
Example:
Consider the following:
SplitType = TypeVar("SplitType")
@dataclass(frozen=True)
class DatasetConfig(Generic[SplitType]):
"""A class specifying the common fields used across all datasets."""
split: SplitType
@dataclass(frozen=True)
class FirstDatasetConfig(DatasetConfig[FirstDatasetSplit]):
"""A class specifying fields for the first dataset."""
version: str
@dataclass(frozen=True)
class SecondDatasetConfig(DatasetConfig[SecondDatasetSplit]):
"""A class specifying fields for the second dataset."""
features: List[str]
class FirstDatasetSplit(Enum):
TRAIN: "train"
TEST: "test"
CHALLENGE: "challenge"
class FirstDatasetSplit(Enum):
TRAIN: "train"
VAL: "val"
TEST: "test"
DEV: "testdev"
The general problem we aim to solve in this case is that we want to define shared fields in a common superclass, but the type of that field is often dependent on the subclass implementation. In the example above, we know that all datasets have a split attribute, however the values that these can take are different depending on the type of dataset.
A workaround for this is to perform checks in a __post_init__ hook, to ensure the value of split is valid, however this often means working with less rigid types like strings instead of enums.
Overview:
Serialiasation support for generic types would allow for serialisation of more complex nested classes/dataclasses
Example:
Consider the following:
The general problem we aim to solve in this case is that we want to define shared fields in a common superclass, but the type of that field is often dependent on the subclass implementation. In the example above, we know that all datasets have a
split
attribute, however the values that these can take are different depending on the type of dataset.A workaround for this is to perform checks in a
__post_init__
hook, to ensure the value ofsplit
is valid, however this often means working with less rigid types like strings instead of enums.