[Bug?]: Dev server builds break after multiple consecutive filesystem saves

[shivghai]

What's not working?

Happening to everyone on my team.

I don't know if this also happens with the web side [Edit - it does not], but if you save an API-side file consecutively (>10x times), the build loop for the dev server breaks and after this state, any changes to the api side aren't reflected in the running code.

This makes for a VERY VERY VERY painful dev experience.

The first error I see on server logs is this:

After that, any changes on the api side aren't reflected and we have to kill the server to have those changes be reflected. The server still runs with the old code.

This is very very consistent. I think this may happen with large Redwood apps (see https://github.com/redwoodjs/redwood/issues/9237 for another example of an issue that my team saw first, presumably because we actually have a large production app).

How do we reproduce the bug?

Save a file (I can 100% repro with API side changes) multiple times consecutively

What's your environment? (If it applies)

yarn rw info:

  System:
    OS: macOS 13.6
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.11.1 - /private/var/folders/8y/99byt5ms31d1z8jgjfmyvcg80000gn/T/xfs-018abefd/node
    Yarn: 3.2.1 - /private/var/folders/8y/99byt5ms31d1z8jgjfmyvcg80000gn/T/xfs-018abefd/yarn
  Databases:
    SQLite: 3.39.5 - /usr/bin/sqlite3
  Browsers:
    Chrome: 126.0.6478.115
    Safari: 16.6
  npmPackages:
    @redwoodjs/auth-auth0-setup: 7.5.0 => 7.5.0 
    @redwoodjs/cli-data-migrate: 7.5.0 => 7.5.0 
    @redwoodjs/core: 7.5.0 => 7.5.0 
  redwood.toml:
    [web]
      sourceMap = true
      title = "Ultralight Labs"
      bundler = "webpack"
      port = "${PORT:8910}"
      host = "0.0.0.0"
      apiUrl = "/api"
      includeEnvironmentVariables = [
           REDACTED
      ] # any ENV vars that should be available to the web side, see https://redwoodjs.com/docs/environment-variables#web
    [api]
      port = "${PORT:8911}"
      host = "0.0.0.0"
    [browser]
      open = false
    [generate]
      tests = false
      stories = false
    [notifications]
      versionUpdates = ["latest"]

Are you interested in working on this?

• [x ] I'm interested in working on this

[dthyresson]

Hi @shivghai turns out that @orta has similar experiences with his larger app as well ... and has invested some time in implementing hot module reloading (HMR) on the api/graphql side (still a WIP). He can provide more context, but perhaps we can offer a better experience soon.

[shivghai]

@dthyresson Is there a thread I can follow? What do you recommend as next steps?

[shivghai]

I'm happy to help with the implementation too, this is painful enough

[orta]

No issue/thread to follow (https://github.com/redwoodjs/redwood/issues/9738 relates because it's been my problem), I've mainly been prototyping outside of Redwood but the current runtime mechanics can be switched to remove the build step by moving to vite via: vite-node

Looking roughly like:

vite-node --watch src/server.ts

const schemas = import.meta.glob("./graphql/**/*.sdl.ts", { eager: true })
const services = import.meta.glob("./services/**/*.ts", { eager: true })

const fastify = Fastify({ logger: true })

const { yoga, logger } = createGraphQLYoga({
  allowGraphiQL: true,
  loggerConfig: {
    logger: createLogger({}),
  },
  services: services as any,
  sdls: schemas as any,
})

// [...]

function killServer() {
  if (!_fastify) return

  _fastify.close(() => {
    console.error("Reset graphql")
  })
}

// @ts-ignore
if (import.meta.hot) {
  // @ts-ignore
  const hot = import.meta.hot
  hot.on("vite:beforeFullReload", () => {
    console.log("HMR: full reload")
    killServer()
  })

  hot.dispose(() => {
    console.log("HMR: dispose")
    killServer()
  })
}

At a guess, to pull this off in Redwood would require:

• understanding how to safely stop using the current dev server
• figure out a watcher for files which need updating, that are currently in the build process
• make a codemod for changing the users user-land graphql function code to use vite-node compatible import syntax instead of the babel modifications

[dthyresson]

Thanks @orta !

@Josh-Walker-GM and @Tobbe - any thoughts on how we might get api/gql side hmr in dev server?

[orta]

I have a more fleshed out working version of HMR in https://github.com/orta/redwood-vite-api