Feature Request: Allow custom scalar types to be configured for jsonb fields

[jamshally]

Desired Outcome/Functionality When using jsonb fields to store json data, it would be great to be able to describe the expected json contents with a custom type. This way, the consuming client can have access to the type information for this field.

Context I am using graphql-code-generator to create TypeScript types from the Hasura schema. I am looking for a way to create strong typing for the contents of jsonb fields

What Else Has Been Tried I have looked for a simple way to rewriting/overriding the field type for the client schema on the client side, prior to the codegen step. However, I have yet found a straight forward mechanism to achieve this

Anything Else? I created a couple of graphql-code-generator plugins to suppor hasura development (https://github.com/ahrnee/graphql-codegen-hasura), and am looking to find a solution to use with this project

[tsnobip]

would love to have this too, using JSONB for different explicit types would make it so much safer and easy to use with type-safe graphQL clients.

[ferm10n]

For future surfers, here's my attempt at using graphql-codegen custom schema loader to implement this:

EDIT: I have a better solution now, below

I have a table app_devices_lnk with a config column of jsonb type. My goal was to make the types for it more narrow.

schema:
  - http://localhost:8080/v1/graphql: # adjust as needed
      headers:
        x-hasura-admin-secret: adminpassword # adjust as needed
      loader: ./schema-loader.js

const { loadSchema } = require('@graphql-tools/load');
const { mergeTypeDefs } = require('@graphql-tools/merge');
const { gql } = require('graphql-tag');
/**
 * merges custom definitions into the schema defined by hasura, overwriting the generic jsonb types
 */
module.exports = async (schemaString, config) => {
    const hasuraSchema = await loadSchema(schemaString, config);

    return mergeTypeDefs([
        // overrides must come first!
        gql` # put your overrides here
            type app_devices_lnk_config_dhl {
                nodeId: String!
                version: String!
                group: String!
                deviceId: String!
            }

            type app_devices_lnk_config_mqtt {
                url: String!
                username: String!
                password: String!
            }

            type app_devices_lnk_config {
                dhl: app_devices_lnk_config_dhl
                mqtt: app_devices_lnk_config_mqtt
            }

            type app_devices_lnk {
                config: app_devices_lnk_config!
            }
        `,
        hasuraSchema,
    ], {
        ignoreFieldConflicts: true,
    });
};

@jamshally

My only gripes with this method are:

• if I ever modify the table or column, this will become an orphan type.
• I don't like having to specify a new type with a global name for every nested object in every jsonb type... maybe I'm too much of a noob with GQL though and don't know a better way to do this. (if anyone does, please end my suffering and show me the way!)

[Brian-Azizi]

Are there any plans to support this natively within Hasura? It would add a lot of safety and improve the dev experience massively when using JSONB types

[zwily]

For future surfers, here's my attempt at using graphql-codegen custom schema loader to implement this:

I have a table app_devices_lnk with a config column of jsonb type. My goal was to make the types for it more narrow.

@ferm10n I tried this and it worked great for injecting the type, however now it complains in my query about not selecting the subfields for the jsonb type. But if I add those subfield selectors in the query, of course Hasura ends up rejecting it with "[GraphQL] unexpected subselection set for non-object field". How did you get past this?

[ferm10n]

@zwily It's been a while since I posted that and my memory is a bit foggy but I'm pretty sure I abandoned that approach because of the issue you're describing :/

HOWEVER! I've learned a bit more about codegen since then so I took another swing at the problem. If you're just trying to get better intellisense in your typescript, I think this will work for ya:

Start with defining schema overrides in a graphql file

# schema-overrides.graphql
scalar mes_app_devices_lnk_config # define a custom scalar type

type mes_app_devices_lnk {
  config: mes_app_devices_lnk_config!
}

type mes_app_devices_lnk_insert_input {
  config: mes_app_devices_lnk_config!
}

Now the codegen.yml. We don't need a schema-loader.js, because specifying multiple schemas automerges if you specify ignoreFieldConflicts: true

# codegen.yml
schema: # no need for schema loader. specifying multiple schemas automagically merge
  - schema-overrides.graphql
  - http://localhost:8080/v1/graphql: # adjust as needed
      headers:
        x-hasura-admin-secret: adminpassword # adjust as needed
config:
  ignoreFieldConflicts: true # required for merge-and-override of schemas
generates:
  ./db-types.ts: # or wherever you want your types
    plugins:
      - add: # inject an import to where the REAL type is
        content: |
          import { AppDevicesLnkCfg } './override-types.ts';
      - typescript:
          scalars: # associate your TS types with your custom scalar types that overrode the json type
            app_devices_lnk_config: AppDevicesLnkCfg

Then, you can make your override-types.ts whatever you want

// override-types.ts
export type AppDevicesLnkCfg = {
  dhl?: {
    nodeId: string;
    version: string;
    group: string;
    deviceId: string;
  };
  mqtt?: {
    url: string;
    username: string;
    password: string;
  }
}

This way when you query, you won't anger hasura by specifying subfields... this only affects your typescript types.

[ferm10n]

I totally forgot about the comment I made here, and I find it kinda amusing how I was still trying (and failing) to get custom types to work for json... but then thanks to your comment reminding me of this thread, I feel like I was able to finally put all the pieces together 😂 so thank you!!

[zwily]

Wow @ferm10n, thank you! This will be a huge help. Getting my responses typed will be amazing.

Hopefully one day Hasura will give us a more elegant way to type (and enforce types going into the db) on jsonb types, but until then, this is a big help.

[ferm10n]

@zwily If you're interested in enforcing the types going into the db, you should check out postgres domains!! This person tried it and ran into the same problem described here.

It basically allows you to extend a base postgres type and add check constraints! Hasura supports this.... unless it's used in a function input with a schema other than public.

[zwily]

@ferm10n Very interesting... Writing a CHECK to verify types on anything other than a pretty simple object seems like it would be pretty hairy though. Thanks for the links!

[deathemperor]

@zwily It's been a while since I posted that and my memory is a bit foggy but I'm pretty sure I abandoned that approach because of the issue you're describing :/

HOWEVER! I've learned a bit more about codegen since then so I took another swing at the problem. If you're just trying to get better intellisense in your typescript, I think this will work for ya:

Start with defining schema overrides in a graphql file

# schema-overrides.graphql
scalar mes_app_devices_lnk_config # define a custom scalar type

type mes_app_devices_lnk {
  config: mes_app_devices_lnk_config!
}

type mes_app_devices_lnk_insert_input {
  config: mes_app_devices_lnk_config!
}

Now the codegen.yml. We don't need a schema-loader.js, because specifying multiple schemas automerges if you specify ignoreFieldConflicts: true

# codegen.yml
schema: # no need for schema loader. specifying multiple schemas automagically merge
  - schema-overrides.graphql
  - http://localhost:8080/v1/graphql: # adjust as needed
      headers:
        x-hasura-admin-secret: adminpassword # adjust as needed
config:
  ignoreFieldConflicts: true # required for merge-and-override of schemas
generates:
  ./db-types.ts: # or wherever you want your types
    plugins:
      - add: # inject an import to where the REAL type is
        content: |
          import { AppDevicesLnkCfg } './override-types.ts';
      - typescript:
          scalars: # associate your TS types with your custom scalar types that overrode the json type
            app_devices_lnk_config: AppDevicesLnkCfg

Then, you can make your override-types.ts whatever you want

// override-types.ts
export type AppDevicesLnkCfg = {
  dhl?: {
    nodeId: string;
    version: string;
    group: string;
    deviceId: string;
  };
  mqtt?: {
    url: string;
    username: string;
    password: string;
  }
}

This way when you query, you won't anger hasura by specifying subfields... this only affects your typescript types.

When I apply this solution I ran into an issue when my scalar type is overwrite by json by resulting type final type still jsonb. any idea how to control the merging order?

[ranneyd]

@zwily It's been a while since I posted that and my memory is a bit foggy but I'm pretty sure I abandoned that approach because of the issue you're describing :/ HOWEVER! I've learned a bit more about codegen since then so I took another swing at the problem. If you're just trying to get better intellisense in your typescript, I think this will work for ya: Start with defining schema overrides in a graphql file

# schema-overrides.graphql
scalar mes_app_devices_lnk_config # define a custom scalar type

type mes_app_devices_lnk {
  config: mes_app_devices_lnk_config!
}

type mes_app_devices_lnk_insert_input {
  config: mes_app_devices_lnk_config!
}

Now the codegen.yml. We don't need a schema-loader.js, because specifying multiple schemas automerges if you specify ignoreFieldConflicts: true

# codegen.yml
schema: # no need for schema loader. specifying multiple schemas automagically merge
  - schema-overrides.graphql
  - http://localhost:8080/v1/graphql: # adjust as needed
      headers:
        x-hasura-admin-secret: adminpassword # adjust as needed
config:
  ignoreFieldConflicts: true # required for merge-and-override of schemas
generates:
  ./db-types.ts: # or wherever you want your types
    plugins:
      - add: # inject an import to where the REAL type is
        content: |
          import { AppDevicesLnkCfg } './override-types.ts';
      - typescript:
          scalars: # associate your TS types with your custom scalar types that overrode the json type
            app_devices_lnk_config: AppDevicesLnkCfg

Then, you can make your override-types.ts whatever you want

// override-types.ts
export type AppDevicesLnkCfg = {
  dhl?: {
    nodeId: string;
    version: string;
    group: string;
    deviceId: string;
  };
  mqtt?: {
    url: string;
    username: string;
    password: string;
  }
}

This way when you query, you won't anger hasura by specifying subfields... this only affects your typescript types.

When I apply this solution I ran into an issue when my scalar type is overwrite by json by resulting type final type still jsonb. any idea how to control the merging order?

I'm having the same issue. If I do something like

type mes_app_devices_lnk {
  config2: String!
}

It ADDS it. But I can't get my custom one to OVERRIDE the main one. I've tried reversing the order in the codegen file and removing the extend in the graphql but to no avail.

UPDATE:

I figured out a way. Instead of ignoreFieldConflicts: true you can put onFieldTypeConflict: existing => existing (I'm using codegen.ts not codegen.yml so I don't know if this works in the yaml). As long as the overrides are first in your schema list, this will work.

[deathemperor]

I figured out a way. Instead of ignoreFieldConflicts: true you can put onFieldTypeConflict: existing => existing (I'm using codegen.ts not codegen.yml so I don't know if this works in the yaml). As long as the overrides are first in your schema list, this will work.

Do you mind sharing your codegen.ts config? I'm using ts too but couldn't figure how onFieldTypeConflict works.

[ranneyd]

I figured out a way. Instead of ignoreFieldConflicts: true you can put onFieldTypeConflict: existing => existing (I'm using codegen.ts not codegen.yml so I don't know if this works in the yaml). As long as the overrides are first in your schema list, this will work.

Do you mind sharing your codegen.ts config? I'm using ts too but couldn't figure how onFieldTypeConflict works.

import type { CodegenConfig } from '@graphql-codegen/cli'

const config: CodegenConfig = {
  schema: [
    // https://github.com/hasura/graphql-engine/issues/3451#issuecomment-1266245636
    '../../hasura/schema-overrides.graphql',
    {
      ...redacted...
    },
  ],
  config: {
    // https://github.com/hasura/graphql-engine/issues/3451#issuecomment-1819859763
    onFieldTypeConflict: (existing: unknown) => existing,
  },
  documents: [
    '../*/src/**/*.graphql',
  ],
  generates: {
    './src/gql/generated': {
      preset: 'gql-tag-operations-preset',
      config: {
        maybeValue: 'T | undefined',
        scalars: {
          uuid: 'string',
          json: 'object',
          MyCustomType: 'myCustomLocation#MyCustomType',
        },
      },
    },
  },
}

export default config

[deathemperor]

I was unable to apply the workaround with near-file-operation, this trick helps solve it:

config: {
  scalars: { RawOcrData: 'Types.Scalars["RawOcrData"]["output"]' },
}

RawOcrData is the type we define in the types.ts section