vs-code CPU 100% after some time in Feathers / Typebox schema

[tvld]

As discussed on Discord, after some time of working in Feathers 5.0.5 with TypeBox, I get a CPU overload in vs-codium. Closing the tab with the schema file brings it back to normal. I work in two panes: left the hooks; right the schema files. The problem appears when I work on a hook. But closing the hook file tab does not solve it. I must close the tab with the schema.

I am terribly sorry that I have no more ideas or relevant information...

[tvld]

For what it is worth, here is my users.schema.ts file:

// For more information about this file see https://dove.feathersjs.com/guides/cli/service.schemas.html

import { resolve, virtual } from '@feathersjs/schema'
import { Type, getValidator, querySyntax, ObjectIdSchema, type Static } from '@feathersjs/typebox'
import { passwordHash } from '@feathersjs/authentication-local'
import { ObjectId } from 'mongodb'
import type { HookContext } from '../../declarations'
import { dataValidator, queryValidator } from '../../schemas/validators'
import { type Locale, LocaleSchema } from '../../schemas/locale'

import {
  AccountSchema,
  DateSchema,
  NotificationSubscriptionSchema,
  SexSchema,
  ShortIsoTime,
  TagsSchema,
  ImageFileNameSchema,
  imagePath,
  gravatarUrl
} from '../../schemas'

import { defaultVal, trimVal, ownerOnly } from '../../schemas/resolver-helpers'

// Main data model schema

const NotificationSchema = Type.Object({
  pushEnabled: Type.Boolean(),
  emailEnabled: Type.Boolean(),
  bligsyTelegramEnabled: Type.Boolean(),
  dayOfWeek: Type.Array(Type.Boolean(), { minItems: 7, maxItems: 7 }), // 0 = Sunday
  shortIsoTime: ShortIsoTime, // time of day such as '03:45' or '20:15', normalized to UTC
  pushSubscription: Type.Optional(NotificationSubscriptionSchema)
})

export type Notification = Static<typeof NotificationSchema>

export const defaultNotificationSchedule: Notification = {
  pushEnabled: false,
  emailEnabled: false,
  bligsyTelegramEnabled: false,
  dayOfWeek: [false, true, true, true, true, true, false],
  shortIsoTime: '18:45'
}

// Email confirm verification object for email & passwords
const ConfirmSchema = Type.Union([
  Type.Object({
    randomCode: Type.Optional(Type.String()),
    codeExpiresAt: Type.Optional(Type.String({ format: 'date-time' }))
  }),
  Type.Null()
])

// Main data model schema
export const userSchema = Type.Object(
  {
    _id: ObjectIdSchema(),
    teamIds: Type.Optional(Type.Array(ObjectIdSchema())),
    teams: Type.Optional(Type.Array(Type.Object({}))), // virtual, populated by resolver
    nrOwnedTeams: Type.Optional(Type.Number()), // virtual resolved, admin only
    nrExperiences: Type.Optional(Type.Number()), // virtual resolved, admin only
    nrAchievements: Type.Optional(Type.Number()), // virtual resolved, admin only

    email: Type.String({ format: 'email' }),
    emailNew: Type.Optional(Type.Union([Type.String({ format: 'email' }), Type.Literal('')])),
    emailNewConfirm: Type.Optional(ConfirmSchema),

    password: Type.String(),
    passwordNew: Type.Optional(Type.String()),
    passwordNewConfirm: Type.Optional(ConfirmSchema),

    isDeleted: Type.Optional(Type.Boolean()), 
    isSuperAdmin: Type.Optional(Type.Boolean()), 
    isAdmin: Type.Optional(Type.Boolean()), 

    termsAgreed: Type.Optional(Type.Boolean()),
    newsLetterOptIn: Type.Optional(Type.Boolean()),
    locale: Type.Optional(LocaleSchema),

    defaultTags: Type.Optional(TagsSchema),

    avatar: Type.Optional(Type.Union([ImageFileNameSchema, Type.String()])),
    avatarFullPath: Type.Optional(Type.String()), // virtual resolved, includes https://

    displayName: Type.Optional(Type.String({ minLength: 2, maxLength: 50 })),
    lastName: Type.Optional(Type.String({ maxLength: 80 })),
    firstName: Type.Optional(Type.String({ maxLength: 80 })),
    sex: Type.Optional(SexSchema),
    company: Type.Optional(Type.String({ maxLength: 100 })),
    jobTitle: Type.Optional(Type.String({ maxLength: 80 })),
    linkedIn: Type.Optional(Type.String({ format: 'uri', maxLength: 250 })),

    resume: Type.Optional(Type.String({ maxLength: 2000 })), 
    includeResume: Type.Optional(Type.Boolean()),

    privatePersonaId: Type.Optional(ObjectIdSchema()), // links to Persona
    privatePersona: Type.Optional(Type.Object({})), // resolved
    workPersonaId: Type.Optional(ObjectIdSchema()),
    workPersona: Type.Optional(Type.Object({})), // resolve

    // account details & billing information
    account: Type.Optional(AccountSchema),

    // notifications
    notificationsEnabled: Type.Optional(Type.Boolean()),
    notificationSchedule: Type.Optional(NotificationSchema),

    // automatic resolved
    createdAt: Type.Optional(DateSchema), // format we use is toISOString()
    updatedAt: Type.Optional(DateSchema),
    loggedInAt: Type.Optional(DateSchema),

    // Mongodb operators
    $addToSet: Type.Optional(Type.Object({})), // { $addToSet: { teamIds: result._id } }
    $push: Type.Optional(Type.Object({})), // { $push: { teamIds: result._id } }
    $pull: Type.Optional(Type.Object({})) // { $pull: { teamIds: result._id } }
  },
  { $id: 'User', additionalProperties: false }
)
export type User = Static<typeof userSchema>
export const userValidator = getValidator(userSchema, dataValidator)
export const userResolver = resolve<User, HookContext>({
  // locale: async (value, data, { provider }) => value || 'en',
  passwordNewConfirm: async (value, data, { provider }) => (provider ? undefined : value),
  emailNewConfirm: async (value, data, { provider }) => (provider ? undefined : value),
  privatePersona: async (value, data, { app, params, provider }) => {
    if (data.privatePersonaId) {
      const response: any = await app
        .service('personas') //
        .find({ query: { _id: data.privatePersonaId } })
      if (response && response.data) return response.data[0] || {}
    }
    return {} as Object[] // required !!
  },
  workPersona: async (value, data, { app, params, provider }) => {
    if (data.workPersonaId) {
      const response: any = await app
        .service('personas') //
        .find({ query: { _id: data.workPersonaId } })
      if (response && response.data) return response.data[0] || {}
    }
    return {} as Object[] // required !!
  }
})

export const userExternalResolver = resolve<User, HookContext>({
  password: async () => undefined, // hide password externally
  sex: defaultVal('O'),
  account: defaultVal({ address: {} }),
  notificationSchedule: defaultVal(defaultNotificationSchedule),

  avatarFullPath: virtual(async (data, context) =>
    data.avatar && data._id
      ? imagePath(data.avatar, data._id?.toString(), 'xa')
      : await gravatarUrl(data._id?.toString())
  ),
  teamIds: defaultVal([]),

  teams: virtual(async (userData, { params, app }) => {
    if (params.provider && userData?.teamIds && userData.teamIds?.length > 0) {
      const response: any = await app
        .service('teams') //
        .find({ query: { _id: { $in: userData.teamIds.map((id: any) => id.toString()) } } })
      if (response && response.data) return response.data
    }
    return [] as Object[] // required !!
  }),

  // adminOnly
  nrOwnedTeams: virtual(async (userData, { params, app }) => {
    if (params.provider && userData?._id && params.user?.isAdmin) {
      const response: any = await app
        .service('teams') //
        .find({ query: { $limit: 0, userId: userData._id.toString() } })
      if (response && response.total) return response.total
      return 0
    }
  }),
  nrExperiences: virtual(async (userData, { params, app }) => {
    if (params.provider && userData?._id && params.user?.isAdmin) {
      const response: any = await app
        .service('experiences') //
        .find({
          query: {
            $limit: 0,
            userId: userData._id.toString(),
            achievement: false
          }
        })
      if (response && response.total) return response.total
      return 0
    }
  }),
  nrAchievements: virtual(async (userData, { params, app }) => {
    if (params.provider && userData?._id && params.user?.isAdmin) {
      const response: any = await app
        .service('experiences') //
        .find({
          query: {
            $limit: 0,
            userId: userData._id.toString(),
            achievement: true
          }
        })
      if (response && response.total) return response.total
      return 0
    }
  })
})

// ** CREATE
export const userDataSchema = Type.Omit(userSchema, ['_id', 'updatedAt', 'createdAt'], {
  $id: 'UserData',
  additionalProperties: false
})
export type UserData = Static<typeof userDataSchema>
export const userDataValidator = getValidator(userDataSchema, dataValidator)
export const userDataResolver = resolve<User, HookContext>({
  password: passwordHash({ strategy: 'local' }),
  email: async (value, user, { params, method }) => value?.toLowerCase().trim(),
  teamIds: async () => [],
  termsAgreed: defaultVal(false),
  newsLetterOptIn: defaultVal(true),
  locale: defaultVal('en' as Locale),
  displayName: trimVal(),
  lastName: trimVal(),
  firstName: trimVal(),
  company: trimVal(),
  jobTitle: trimVal(),
  linkedIn: trimVal(),

  defaultTags: async (value, user, { app }) => (!value ? app.get('defaultTags') : value?.sort()),

  createdAt: async (value, user, context) => new Date().toISOString(),
  updatedAt: async (value, user, context) => new Date().toISOString()
})

// Schema for updating existing users
export const userPatchSchema = Type.Partial(userSchema, {
  $id: 'UserPatch'
})

export type UserPatch = Static<typeof userPatchSchema>
export const userPatchValidator = getValidator(userPatchSchema, dataValidator)
export const userPatchResolver = resolve<UserPatch, HookContext>({
  passwordNew: passwordHash({ strategy: 'local' }),

  email: async (value, user, { params, method }) => {
    if (!params.provider) return value?.toLowerCase().trim()
  },
  // cache enables; for easy CRON search
  notificationsEnabled: async (value, user, { params, method }) =>
    user?.notificationSchedule?.pushEnabled ||
    user?.notificationSchedule?.emailEnabled ||
    user?.notificationSchedule?.bligsyTelegramEnabled,

  emailNew: async (value, user, { params, method }) => value?.toLowerCase().trim(),

  teamIds: async (value, data, context) => {
    if (!context.params.provider) return value
  },
  // Leave team with { $pull: { teamIds: team._id } }
  // but front end does not convert, so we do it here
  $pull: async (value, data, { params }) => {
    const { user, provider } = params
    if (provider && value && 'teamIds' in value) {
      value.teamIds = Array.isArray(value.teamIds)
        ? value.teamIds.map((i) => new ObjectId(i))
        : new ObjectId(value.teamIds as string)
    }
    return value
  },
  defaultTags: async (value, user, { app, method }) => value?.sort(),

  updatedAt: async (value, user, context) => new Date().toISOString()
})

// Schema for allowed query properties
export const userQueryProperties = Type.Omit(userSchema, [])

export const userQuerySchema = Type.Intersect(
  [
    querySyntax(
      Type.Intersect([
        userQueryProperties,
        Type.Object({
          'notificationSchedule.pushEnabled': Type.Optional(Type.Boolean()),
          'notificationSchedule.dayOfWeek.0': Type.Optional(Type.Boolean()),
          'notificationSchedule.dayOfWeek.1': Type.Optional(Type.Boolean()),
          'notificationSchedule.dayOfWeek.2': Type.Optional(Type.Boolean()),
          'notificationSchedule.dayOfWeek.3': Type.Optional(Type.Boolean()),
          'notificationSchedule.dayOfWeek.4': Type.Optional(Type.Boolean()),
          'notificationSchedule.dayOfWeek.5': Type.Optional(Type.Boolean()),
          'notificationSchedule.dayOfWeek.6': Type.Optional(Type.Boolean()),
          'notificationSchedule.shortIsoTime': Type.Optional(Type.String())
        })
      ]),
      {
        email: {
          $regex: Type.String(),
          $options: Type.Optional(Type.String())
        },
        displayName: {
          $regex: Type.String(),
          $options: Type.Optional(Type.String())
        },
        firstName: {
          $regex: Type.String(),
          $options: Type.Optional(Type.String())
        },
        lastName: {
          $regex: Type.String(),
          $options: Type.Optional(Type.String())
        },
        company: {
          $regex: Type.String(),
          $options: Type.Optional(Type.String())
        },
        // see
        teamIds: {
          $in: Type.Array(ObjectIdSchema())
          // $in: Type.Array(Type.String({ objectid: true }))
        }
        // 'notificationSchedule.pushEnabled': Type.Boolean()
      }
    ),
    Type.Object(
      {
        teamIds: Type.Optional(
          Type.Object(
            {
              $in: Type.Array(ObjectIdSchema())
            },
            { additionalProperties: false }
          )
        )
      },
      { additionalProperties: false }
    )
  ],
  { additionalProperties: false }
)
export type UserQuery = Static<typeof userQuerySchema>
export const userQueryValidator = getValidator(userQuerySchema, queryValidator)
export const userQueryResolver = resolve<UserQuery, HookContext>({
  $select: async (value, query, { params, method, path }) => {
    const { user, provider } = params
    // auth. users are only allowed to see part of data of other users
    return provider && method === 'find' && params.user && !params.user.isAdmin
      ? [
          '_id',
          'avatar',
          'avatarFullPath',
          'company',
          'createdAt',
          'displayName',
          'email',
          'firstName',
          'jobTitle',
          'lastName',
          'sex',
          'linkedIn'
        ]
      : value
  }
})

[sinclairzx81]

@tvld Hi! Thanks for the repro.

Having a quick look at this, I can only reasonably replicate the User type (excluding Feathers specific resolvers). Here's a quick updated version removing the Feathers specifics.

TypeScript Link Here

import { Type, type Static } from '@sinclair/typebox'

// ------------------------------------------------------------------------
// Fake Definitions
// ------------------------------------------------------------------------
const ObjectIdSchema = () => Type.Object({})
const ShortIsoTime = Type.String()
const NotificationSubscriptionSchema = Type.Object({ x: Type.Number() })
const LocaleSchema = Type.Object({ x: Type.Number() })
const TagsSchema = Type.Object({ x: Type.Number() })
const ImageFileNameSchema = Type.Object({ x: Type.Number() })
const SexSchema = Type.Object({ x: Type.Number() })
const AccountSchema = Type.Object({ x: Type.Number() })
const DateSchema = Type.Object({ x: Type.Number() })

// ------------------------------------------------------------------------
// Main data model schema
// ------------------------------------------------------------------------
const NotificationSchema = Type.Object({
  pushEnabled: Type.Boolean(),
  emailEnabled: Type.Boolean(),
  bligsyTelegramEnabled: Type.Boolean(),
  dayOfWeek: Type.Array(Type.Boolean(), { minItems: 7, maxItems: 7 }), // 0 = Sunday
  shortIsoTime: ShortIsoTime, // time of day such as '03:45' or '20:15', normalized to UTC
  pushSubscription: Type.Optional(NotificationSubscriptionSchema)
})

export type Notification = Static<typeof NotificationSchema>

export const defaultNotificationSchedule: Notification = {
  pushEnabled: false,
  emailEnabled: false,
  bligsyTelegramEnabled: false,
  dayOfWeek: [false, true, true, true, true, true, false],
  shortIsoTime: '18:45'
}
const ConfirmSchema = Type.Union([
  Type.Object({
    randomCode: Type.Optional(Type.String()),
    codeExpiresAt: Type.Optional(Type.String({ format: 'date-time' }))
  }),
  Type.Null()
])
export const userSchema = Type.Object(
  {
    _id: ObjectIdSchema(),
    teamIds: Type.Optional(Type.Array(ObjectIdSchema())),
    teams: Type.Optional(Type.Array(Type.Object({}))), // virtual, populated by resolver
    nrOwnedTeams: Type.Optional(Type.Number()), // virtual resolved, admin only
    nrExperiences: Type.Optional(Type.Number()), // virtual resolved, admin only
    nrAchievements: Type.Optional(Type.Number()), // virtual resolved, admin only

    email: Type.String({ format: 'email' }),
    emailNew: Type.Optional(Type.Union([Type.String({ format: 'email' }), Type.Literal('')])),
    emailNewConfirm: Type.Optional(ConfirmSchema),

    password: Type.String(),
    passwordNew: Type.Optional(Type.String()),
    passwordNewConfirm: Type.Optional(ConfirmSchema),

    isDeleted: Type.Optional(Type.Boolean()), 
    isSuperAdmin: Type.Optional(Type.Boolean()), 
    isAdmin: Type.Optional(Type.Boolean()), 

    termsAgreed: Type.Optional(Type.Boolean()),
    newsLetterOptIn: Type.Optional(Type.Boolean()),
    locale: Type.Optional(LocaleSchema),

    defaultTags: Type.Optional(TagsSchema),

    avatar: Type.Optional(Type.Union([ImageFileNameSchema, Type.String()])),
    avatarFullPath: Type.Optional(Type.String()), // virtual resolved, includes https://

    displayName: Type.Optional(Type.String({ minLength: 2, maxLength: 50 })),
    lastName: Type.Optional(Type.String({ maxLength: 80 })),
    firstName: Type.Optional(Type.String({ maxLength: 80 })),
    sex: Type.Optional(SexSchema),
    company: Type.Optional(Type.String({ maxLength: 100 })),
    jobTitle: Type.Optional(Type.String({ maxLength: 80 })),
    linkedIn: Type.Optional(Type.String({ format: 'uri', maxLength: 250 })),

    resume: Type.Optional(Type.String({ maxLength: 2000 })), 
    includeResume: Type.Optional(Type.Boolean()),

    privatePersonaId: Type.Optional(ObjectIdSchema()), // links to Persona
    privatePersona: Type.Optional(Type.Object({})), // resolved
    workPersonaId: Type.Optional(ObjectIdSchema()),
    workPersona: Type.Optional(Type.Object({})), // resolve

    // account details & billing information
    account: Type.Optional(AccountSchema),

    // notifications
    notificationsEnabled: Type.Optional(Type.Boolean()),
    notificationSchedule: Type.Optional(NotificationSchema),

    // automatic resolved
    createdAt: Type.Optional(DateSchema), // format we use is toISOString()
    updatedAt: Type.Optional(DateSchema),
    loggedInAt: Type.Optional(DateSchema),

    // Mongodb operators
    $addToSet: Type.Optional(Type.Object({})), // { $addToSet: { teamIds: result._id } }
    $push: Type.Optional(Type.Object({})), // { $push: { teamIds: result._id } }
    $pull: Type.Optional(Type.Object({})) // { $pull: { teamIds: result._id } }
  },
  { $id: 'User', additionalProperties: false }
)
export type User = Static<typeof userSchema>

// -----------------------------------------------------------------------
// Test function
// -----------------------------------------------------------------------
function test(value: User) {
  // test for inference performance here 
}

Suggestions

• Try move the User definition into it's own file (including only the above) and keep the resolvers in a separate file. This just to further narrow the problem down (as the performance inference may relate to the querySyntax). if you observe the User schema definition file causing problems still, I will do some further investigations (but this would perhaps point more to an editor issue in vs-codium)
• For the Type.Optional(), you can omit these and wrap the userSchema in a Type.Partial(userSchema). This may yield more efficient inference as many field level optionals may be more challenging to infer (this is unlikely, but may be easier to edit)
• For the intersected types (relating to notificationSchedule.pushEnabled object), try pull these types out into their own types on separate lines external to the intersect. TypeScript can have a better time if it can evaluate and cache types that span different lines (this is a caching behavior of TypeScript). There has been some novel cases where this has helped in other large inference scenarios, it may help in this case.

Thanks again for the repro, I can't see anything immediate in TypeBox specifically that would cause this issue (and the TypeScript Link seems to work ok which is running fairly top heavy in a browser context). I think by splitting the User definition and the resolvers that would help to pinpoint what specifically is causing the CPU to throttle. My initial thoughts are probably the Intersected querySyntax but without being familiar with the implementation, it's hard to tell.

Hope this helps! S

[tvld]

Thanks for the deep checking. Although they are quite straightforward, apologies for not including the sub-types. It's a good idea to split the file further in schema only and the resolvers; will do. Also, the queryResolver... it might be something. I recall having the same pattern in a couple of other service schemas. And they suffer from the same CPU 100% issue... but as for not I don't know if it's 1:1 related...

I will do some more digging. The trouble in all this is that the issue appears after like half an hour or so. Never immediately... so this is one of those bugs.... ;)))))

[sinclairzx81]

@tvld Hey, no problem :)

Have left my machine on overnight with this reduced schema (as above), haven't noted any issues with CPU utilization. I'm somewhat leaning towards this being a vs-codium issue (though it's very difficult to say with any certainty). The queryResolver could also be at fault also; but even I'd expect the editor to essentially be idle when not in focus.

Most editors (and I assume vs-codium too) would be using the Language Service Protocol (or LSP) to provide intellisense (which is the only thing I can imagine could be causing the issue). A fault here might be the editor exchanging (or throttling) small JSON message payloads between the editor and compiler backend (or language service). I'm not familiar with the context in which the language server runs in vs-codium (I assume it runs as a Web Worker in vscode), but you might be able to bring up the devtools console (Help -> Toggle Developer Tools) and see if you can trace anything in there (it's a long shot, but it might reveal something)

Will close up this issue for now, but if come across anything, let me know (tools for debugging and profiling the editor experience are few and far between). If tracing through devtools helps to highlight the problem, I'd be very interested to know (particularly for debugging issues of this sort in future)

All the best! S

[tvld]

Thanks for the effort. I have been logging the typescript compiler, but that did not give me any clues. I will keep my eye on it of course and let you know if needed. In the meantime, if you in the future like me to test anything specific, I'll be happy to! ))