Serialized fields must be described in 3 different ways.

[pixelzoom]

Encountered while instrumenting equality-explorer for https://github.com/phetsims/equality-explorer/issues/192. And I've encountered this before, but TypeScript has made the problem worse, so I'm finally creating an issue.

There is an undesirable amount of duplication involved in definiting an IOType. For example, let's look at equality-explorer Challenge.ts, where ChallengeIO is defined.

First, there are the fields in the core type. In the case of Challenge, I have 6 fields, defined using constructor assignment shorthand:

export default class Challenge {
...
  public constructor( public readonly x: number,
                      public readonly a: Fraction,
                      public readonly b: Fraction,
                      public readonly m: Fraction,
                      public readonly n: Fraction,
                      public readonly debugDerivation: string ) {

Second, there's the StateObject type, in this case ChallengeStateObject. Here I have the same 6 fields, but they need to be defined as StateObject types (with StringIO being the exception, which is a different problem, see https://github.com/phetsims/tandem/issues/277.)

export type ChallengeStateObject = {
  x: NumberStateObject;
  a: FractionStateObject;
  b: FractionStateObject;
  m: FractionStateObject;
  n: FractionStateObject;
  debugDerivation: string;
};

Third, there's the definition of stateSchema for the IOType, in this case ChallengeIO. Again I have the same 6 fields, but here they need to defined as IOTypes:

  public static readonly ChallengeIO = new IOType<Challenge, ChallengeStateObject>( 'ChallengeIO', {
    stateSchema: {
      x: NumberIO,
      a: Fraction.FractionIO,
      b: Fraction.FractionIO,
      m: Fraction.FractionIO,
      n: Fraction.FractionIO,
      debugDerivation: StringIO
    },
    ...
  } );

So in general, if I have N fields to serialize, I must describe those fields in 3 different ways:

• core type
• StateObject type
• IOType

And as I make changes, I need to make changes in 3 places, and use the correct type in those places. That's a lot of undesirable duplication, and a lot to keep track of.

Is there something that can be done to improve this?

[pixelzoom]

I should also point out that toStateObject and fromStateObject also contain a form of duplication. For ChallengeIO:

  public toStateObject(): ChallengeStateObject {
    return {
      x: NumberIO.toStateObject( this.x ),
      a: Fraction.FractionIO.toStateObject( this.a ),
      b: Fraction.FractionIO.toStateObject( this.b ),
      m: Fraction.FractionIO.toStateObject( this.m ),
      n: Fraction.FractionIO.toStateObject( this.n ),
      debugDerivation: StringIO.toStateObject( this.debugDerivation )
    };
  }

  public static fromStateObject( stateObject: ChallengeStateObject ): Challenge {
    return new Challenge(
      NumberIO.fromStateObject( stateObject.x ),
      Fraction.FractionIO.fromStateObject( stateObject.a ),
      Fraction.FractionIO.fromStateObject( stateObject.b ),
      Fraction.FractionIO.fromStateObject( stateObject.m ),
      Fraction.FractionIO.fromStateObject( stateObject.n ),
      stateObject.debugDerivation
    );
  }

So in general, if I change a field, I need to change it in 5 places, not 3 places.

[samreid]

I took a look at this, and tried to consolidate types. Here is a non-running patch:

```diff Index: js/solveit/model/Challenge.ts IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/solveit/model/Challenge.ts b/js/solveit/model/Challenge.ts --- a/js/solveit/model/Challenge.ts (revision 16bae7574042383a33cea1e4f473fb5070789c25) +++ b/js/solveit/model/Challenge.ts (date 1666624409026) @@ -6,21 +6,27 @@ * @author Chris Malley (PixelZoom, Inc.) */ -import Fraction, { FractionStateObject } from '../../../../phetcommon/js/model/Fraction.js'; +import Fraction from '../../../../phetcommon/js/model/Fraction.js'; import StringUtils from '../../../../phetcommon/js/util/StringUtils.js'; import IOType from '../../../../tandem/js/types/IOType.js'; -import NumberIO, { NumberStateObject } from '../../../../tandem/js/types/NumberIO.js'; -import StringIO, { StringStateObject } from '../../../../tandem/js/types/StringIO.js'; +import NumberIO from '../../../../tandem/js/types/NumberIO.js'; +import StringIO from '../../../../tandem/js/types/StringIO.js'; import equalityExplorer from '../../equalityExplorer.js'; -export type ChallengeStateObject = { - x: NumberStateObject; - a: FractionStateObject; - b: FractionStateObject; - m: FractionStateObject; - n: FractionStateObject; - debugDerivation: StringStateObject; +const stateSchema = { + x: NumberIO, + a: Fraction.FractionIO, + b: Fraction.FractionIO, + m: Fraction.FractionIO, + n: Fraction.FractionIO, + debugDerivation: StringIO +} as const; + +type StateObject> = { + [Key in keyof M]: ReturnType; }; + +export type ChallengeStateObject = StateObject; export default class Challenge { @@ -64,46 +70,17 @@ } ); } - /** - * PhET-iO serialization - */ - public toStateObject(): ChallengeStateObject { - return { - x: NumberIO.toStateObject( this.x ), - a: Fraction.FractionIO.toStateObject( this.a ), - b: Fraction.FractionIO.toStateObject( this.b ), - m: Fraction.FractionIO.toStateObject( this.m ), - n: Fraction.FractionIO.toStateObject( this.n ), - //TODO debugDerivation is for debugging, but will show up in Studio, complete with HTML5 markup. Is it OK? Document in client guide? - debugDerivation: StringIO.toStateObject( this.debugDerivation ) - }; - } - /** * PhET-iO deserialization */ public static fromStateObject( stateObject: ChallengeStateObject ): Challenge { - return new Challenge( - NumberIO.fromStateObject( stateObject.x ), - Fraction.FractionIO.fromStateObject( stateObject.a ), - Fraction.FractionIO.fromStateObject( stateObject.b ), - Fraction.FractionIO.fromStateObject( stateObject.m ), - Fraction.FractionIO.fromStateObject( stateObject.n ), - stateObject.debugDerivation - ); + const coreObject = StateSchema.defaultFromStateObject( stateObject ); + return new Challenge( coreObject.x, coreObject.a, coreObject.b, coreObject.m, coreObject.n, coreObject.debugDerivation ); } public static readonly ChallengeIO = new IOType( 'ChallengeIO', { valueType: Challenge, - stateSchema: { - x: NumberIO, - a: Fraction.FractionIO, - b: Fraction.FractionIO, - m: Fraction.FractionIO, - n: Fraction.FractionIO, - debugDerivation: StringIO - }, - toStateObject: ( challenge: Challenge ) => challenge.toStateObject(), + stateSchema: stateSchema, fromStateObject: ( stateObject: ChallengeStateObject ) => Challenge.fromStateObject( stateObject ) } ); } ```

This patch includes this new type:

type StateObject<M extends Record<string, IOType>> = {
  [Key in keyof M]: ReturnType<M[Key]['toStateObject']>;
};

This takes a StateSchema and gets all of its state types. This works well for terminal/primitive states, but it doesn't handle recursively/nested composite types.

But for cases like this, it allows us to infer that state type from a schema:

export type ChallengeStateObject = StateObject<typeof stateSchema>;

I also noticed StateSchema doesn't support defaultFromStateObject but if implemented, it could be used like so:

  public static fromStateObject( stateObject: ChallengeStateObject ): Challenge {
    const coreObject = StateSchema.defaultFromStateObject( stateObject );
    return new Challenge( coreObject.x, coreObject.a, coreObject.b, coreObject.m, coreObject.n, coreObject.debugDerivation );
  }

After these changes, the central type is the StateSchema. StateObject type is derived from that. Still have to unpack things to call a constructor.

This doesn't seem higher priority that existing milestones, but I wanted to jot some thoughts down. A good next step would be discussion/feedback with @zepumph

[zepumph]

Ok. I actually really like @samreid's idea in the above patch, but I am getting some really weird lack of errors from typescript. I created a strict version that doesn't let IntentionalAny leak into state object types, but it doesn't type check on never as I would have expected. @samreid can you apply the patch and look at that TODO? I'm not really trusting my system type checking right now.

```diff Subject: [PATCH] add a console log, https://github.com/phetsims/phet-io/issues/1944 --- Index: equality-explorer/js/solveit/model/Challenge.ts IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/equality-explorer/js/solveit/model/Challenge.ts b/equality-explorer/js/solveit/model/Challenge.ts --- a/equality-explorer/js/solveit/model/Challenge.ts (revision 89c5e1d86f150c5e7cbab4fdc81868f653272e92) +++ b/equality-explorer/js/solveit/model/Challenge.ts (date 1687883847653) @@ -6,21 +6,38 @@ * @author Chris Malley (PixelZoom, Inc.) */ -import Fraction, { FractionStateObject } from '../../../../phetcommon/js/model/Fraction.js'; +import Fraction from '../../../../phetcommon/js/model/Fraction.js'; import StringUtils from '../../../../phetcommon/js/util/StringUtils.js'; import IOType from '../../../../tandem/js/types/IOType.js'; +import IntentionalAny from '../../../../phet-core/js/types/IntentionalAny.js'; import NumberIO from '../../../../tandem/js/types/NumberIO.js'; import StringIO from '../../../../tandem/js/types/StringIO.js'; import equalityExplorer from '../../equalityExplorer.js'; -export type ChallengeStateObject = { - x: number; - a: FractionStateObject; - b: FractionStateObject; - m: FractionStateObject; - n: FractionStateObject; - debugDerivation: string; + +const stateSchema = { + x: NumberIO, + a: Fraction.FractionIO, + b: Fraction.FractionIO, + m: Fraction.FractionIO, + n: Fraction.FractionIO, + debugDerivation: StringIO +} as const; + +type StateObject> = { + [Key in keyof M]: ReturnType; }; + + +type StateObjectStrict> = { + [Key in keyof M]: IntentionalAny extends ReturnType ? never : ReturnType; +}; + +export type ChallengeStateObject = StateObject; +export type ChallengeStateObjectStrict = StateObjectStrict; + +// type X = ChallengeStateObjectStrict['x']; +// never! Woo export default class Challenge { @@ -64,26 +81,12 @@ } ); } - /** - * PhET-iO serialization - */ - public toStateObject(): ChallengeStateObject { - return { - x: this.x, - a: Fraction.FractionIO.toStateObject( this.a ), - b: Fraction.FractionIO.toStateObject( this.b ), - m: Fraction.FractionIO.toStateObject( this.m ), - n: Fraction.FractionIO.toStateObject( this.n ), - debugDerivation: this.debugDerivation //TODO https://github.com/phetsims/equality-explorer/issues/191 document in client guide? - }; - } - /** * PhET-iO deserialization */ - public static fromStateObject( stateObject: ChallengeStateObject ): Challenge { + public static fromStateObject( stateObject: ChallengeStateObjectStrict ): Challenge { return new Challenge( - stateObject.x, + stateObject.x, // TODO why no type error when passing through never? Fraction.FractionIO.fromStateObject( stateObject.a ), Fraction.FractionIO.fromStateObject( stateObject.b ), Fraction.FractionIO.fromStateObject( stateObject.m ), @@ -92,18 +95,10 @@ ); } - public static readonly ChallengeIO = new IOType( 'ChallengeIO', { + public static readonly ChallengeIO = new IOType( 'ChallengeIO', { valueType: Challenge, - stateSchema: { - x: NumberIO, - a: Fraction.FractionIO, - b: Fraction.FractionIO, - m: Fraction.FractionIO, - n: Fraction.FractionIO, - debugDerivation: StringIO - }, - toStateObject: ( challenge: Challenge ) => challenge.toStateObject(), - fromStateObject: ( stateObject: ChallengeStateObject ) => Challenge.fromStateObject( stateObject ) + stateSchema: stateSchema, + fromStateObject: stateObject => Challenge.fromStateObject( stateObject ) } ); } Index: tandem/js/types/NumberIO.ts IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/tandem/js/types/NumberIO.ts b/tandem/js/types/NumberIO.ts --- a/tandem/js/types/NumberIO.ts (revision 4e14ac8c14514e2af5342a623451732403d22f82) +++ b/tandem/js/types/NumberIO.ts (date 1687883196947) @@ -11,7 +11,7 @@ import IOType from './IOType.js'; import StateSchema from './StateSchema.js'; -const NumberIO = new IOType( 'NumberIO', { +const NumberIO = new IOType( 'NumberIO', { valueType: 'number', documentation: 'IO Type for Javascript\'s number primitive type', toStateObject: _.identity,

[samreid]

@zepumph and I worked on this yesterday and I thought we wrote a message that we were at a good point to check in with @pixelzoom. But I'm not sure if I'm thinking of another issue.

[zepumph]

Basically we asking for feedback on getting rid of one of the three descriptions.

In this pattern we pull out the stateSchema object from the IOType option, and then use it as a const to get the typescript type of the state object for it. I just committed the same change to Vector2 so that you could see another example of the conversion.

[pixelzoom]

@zepumph said:

Basically we asking for feedback on getting rid of one of the three descriptions. ...

I've read the comments several times, looked at the commits, and looked at Vector2IO. I can't decipher which of "the three descriptions" you're proposing to get rid of, or how this affects writing IOTypes in general.