search

pantsbuild / pants

The Pants Build System
https://www.pantsbuild.org
Apache License 2.0
3.35k stars 640 forks source link

Better support for tools with built-in file-watching/hot-reload, for faster dev loops #17414

Open huonw opened 2 years ago

huonw commented 2 years ago

Is your feature request related to a problem? Please describe.

A lot of dev tools (especially in the JS/TS world, #14190) have built-in hot reload in response to file changes. This hot reload is often much faster than starting the whole dev tool from scratch, and can preserve context/state.

For instance, a lot of frontend UI dev work is done via https://storybook.js.org to be able to view, test and iterate on components in isolation, quickly. It seems a common dev flow is to start the storybook server as a long running process (maybe as an NPM script, e.g. "scripts": { "storybook": "start-storybook -p 6006 -s public", ... } in package.json, which can be invoked as npm run storybook), and then edit the code. The changes becomes visible in the storybook interface very soon after via file-watching and hot-reload built in to storybook. Starting storybook from scratch takes much longer (1min or more in our codebase).

(There's other examples too, e.g. our frontend has a start script to start the react development server, that does similar hot-reloading for the whole frontend when the code changes, without needing to start from scratch, or lose (much) state.)

AIUI, pants will typically interrupt a long running process when there's changes, and then start it in a new sandbox. This is a significantly worse dev experience for at least two reasons:

Describe the solution you'd like

A flag similar to restartable that has almost exactly 'opposite' behaviour: instead of interrupting the process and starting in a new sandbox, just copy across any changes into the existing sandbox and let the code running within the sandbox respond to that change. (This includes copying in all transitive changes, such as codegen output changing.)

For instance, in the best case, we might have something like

typescript_sources(name="frontend", sources=["**/*.ts", "**/*.tsx"])
experimental_shell_command(
   name="storybook"
   command="npx start-storybook -p 6006 -s public",
   dependencies=[":frontend"],
   tools=[...],
   update_sandbox_live=True, # NEW FEATURE
)

(A "best case" sketch relying on #16803 (for running the shell command), #16961 (for typescript) and #17405 (related to being able to run the start-storybook command from an external dependency).)

Given this ./pants run path/to:storybook would start a long-running storybook server (for instance, in /tmp/pants/sandbox/123abc), and changes to any of the frontend files would be copied into /tmp/pants/sandbox/123abc directly.

Describe alternatives you've considered

I cannot think of one đŸ˜„

Additional context

Some solution here will be important for us to be able to migrate the JS/TS parts of our monorepo to pants, in addition to the Python parts.

huonw commented 1 year ago

Hm, potentially this may need to be more fine-grained: are there tools where the hot reload monitoring tooling for some files, but not others (e.g. source files vs. other data inputs)? If so, it'd be nifty to be able to tell pants this, so that it does a sandbox update when possible or a full restart if required.

cburroughs commented 8 months ago

As another concrete but Python-y usecase, we use mkdocs which has a mkdocs serve command that will do a quick hot-reload if any of the docs changes. I think most Python-y doc systems (#17414) have something similar.

Hm, potentially this may need to be more fine-grained

A list of the live-update-able dependencies? Or maybe an inclusion/exclusion pair of fields to cover all bases?

huonw commented 1 month ago

Bazel persistent workers are a potentially-related prior art: