Daemon should have been opt in, not opt out with --no-daemon, add config opt out in turbo.json

[AddictArts]

Which project is this feature idea for?

Turborepo

Describe the feature you'd like to request

Add config opt out in turbo.json

{
"$schema": "https://turbo.build/schema.json",
"daemon": false
}

Describe the solution you'd like

A config option to disable by default.

Describe alternatives you've considered

I run with --no-daemon as an alt. This is due to the daemon not really working correctly. It constantly spits out erroneous errors, specifically error Unknown. It clutters the output with confusing messages and doesn't improve performance. I also don't need remote caching. As mentioned the daemon should be opt in and configure, not opt out.

[KevinRouchut]

Absolutely needed !

Currently ugly like this :

[chris-olszewski]

We're about to rework the messaging around daemon so it will no longer be as visible. cc @anthonyshew

Is the noise of the daemon errors the primary thing you're trying to avoid?

[AddictArts]

@chris-olszewski Not completely the noise. I'd like to run turbo with a default --no-daemon. I am sure I can work up a Powershell or nix alias to do it, but IMHO it should be default opt-in not opt-out. Especially when it offers no benefit that I can see or tell. For large teams that may not be the case, but they can opt-in.

I keep trying the daemon and it does not work on my mono-repo. It doesn't cache as it should and I don't have the time to troubleshoot it. I don't have the time to give an example mono-repo so you can see it. Running with --no-daemon is the only way to keep sanity and have accurate meaningful console messages. The daemon just adds confusion for no real benefit.

[timmattison]

Also, on Windows it leaves me with directories I can't delete without killing turbo.exe. And sometimes even after killing turbo.exe the directories still can't be deleted even by an admin. I put --no-daemon everywhere but somehow if I run npm run build with --no-daemon in my package.json instead of running turbo --no-daemon run build it creates these immutable directories again. The only reliably solution I've found is to reboot (or maybe logout) which is silly.

[kostia1st]

I can confirm - I also experience systematic problems with turbo.exe hanging in memory and blocking node_modules from being wiped out whenever I need it, not to mention redundant console messages about "standalone daemon not being found" or alike.

In my view, as a consumer you don't have to know or note these internal hiccups in order to use the tool "as is", and "as is" should better be configured to work frictionlessly from the get go. This is not to discourage or insult anybody working on the tool, nothing like that. I am just wondering why it is the way it is, and if we could do anything about it.

[mpereira]

I started seeing the failed to contact turbod. Continuing in standalone mode: connection to turbo daemon process failed warning after upgrading turbo from 1.10.14 to 1.10.16.

turbo daemon {clean,start,restart} doesn't help. I also tried:

• removing all .turbo directories in the repository
• removing the referenced temporary file under /var/folders

Nothing helped. I also note that the temporary files referenced in the output don't seem to change even though I deleted the directory. A different directory name than a28b2d553bc195a3 exists under /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod:

$ ls /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod
b1f2d416695e7a92

Output from turbo -vvv ... command:

2023-10-21T22:44:35.716+0200 [DEBUG] turborepo_lib::shim: Global turbo version: 1.10.16
2023-10-21T22:44:35.718+0200 [DEBUG] turborepo_lib::shim: Repository Root: <repo_root>
2023-10-21T22:44:35.718+0200 [DEBUG] turborepo_lib::shim: Local turbo path: <repo_root>/node_modules/turbo-darwin-arm64/bin/turbo
2023-10-21T22:44:35.718+0200 [DEBUG] turborepo_lib::shim: Local turbo version: 1.10.16
2023-10-21T22:44:35.718+0200 [DEBUG] turborepo_lib::shim: Currently running turbo is local turbo.
2023-10-21T22:44:35.719+0200 [TRACE] turborepo_lib::execution_state: Found npm as package manager
2023-10-21T22:44:35.719+0200 [DEBUG] turbo: Found go binary at "<repo_root>/node_modules/turbo-darwin-arm64/bin/go-turbo"
2023-10-21T22:44:35.719+0200 [TRACE] turbo: Invoking go binary with {"config":{"apiUrl":null,"loginUrl":null,"teamSlug":null,"teamId":null,"token":null,"signature":null,"preflight":null,"timeout":null,"enabled":null},"global_hash":null,"task_hash_tracker":null,"api_client_config":{"token":null,"team_id":null,"team_slug":null,"api_url":"https://vercel.com/api","use_preflight":false,"timeout":20},"spaces_api_client_config":{"token":null,"team_id":null,"team_slug":null,"api_url":"https://vercel.com/api","use_preflight":false,"timeout":20},"package_manager":"npm","cli_args":{"api":null,"color":false,"cpu_profile":null,"cwd":"<repo_root>","heap":null,"login":null,"no_color":false,"preflight":false,"remote_cache_timeout":null,"team":null,"token":null,"trace":null,"verbosity":3,"test_run":false,"command":{"Run":{"cache_dir":null,"cache_workers":10,"concurrency":null,"continue_execution":false,"dry_run":null,"single_package":false,"filter":["<app>"],"force":null,"framework_inference":true,"global_deps":[],"graph":null,"env_mode":"Infer","ignore":[],"include_dependencies":false,"no_cache":false,"no_daemon":false,"no_deps":false,"output_logs":null,"log_order":"auto","only":false,"parallel":false,"pkg_inference_root":null,"profile":null,"remote_only":false,"scope":[],"since":null,"summarize":null,"log_prefix":"auto","tasks":["dev"],"pass_through_args":[],"experimental_space_id":null,"experimental_rust_codepath":false}}}}
2023-10-21T22:44:35.834+0200 [DEBUG] turbo: build tag: rust
2023-10-21T22:44:35.848+0200 [DEBUG] turbo.TurbodClient: starting turbod binary <repo_root>/node_modules/turbo-darwin-arm64/bin/turbo
2023-10-21T22:44:36.849+0200 [DEBUG] turbo.TurbodClient: starting turbod binary <repo_root>/node_modules/turbo-darwin-arm64/bin/turbo
2023-10-21T22:44:37.850+0200 [DEBUG] turbo.TurbodClient: starting turbod binary <repo_root>/node_modules/turbo-darwin-arm64/bin/turbo
2023-10-21T22:44:38.851+0200 [WARN]  turbo: :
  warning=
  | failed to contact turbod. Continuing in standalone mode: connection to turbo daemon process failed.
  |     To quickly resolve the issue, try running:
  |     - $ turbo daemon clean
  | 
  |     To debug further - please ensure the following:
  |     - the process identified by the pid in the file at /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod/a28b2d553bc195a3/turbod.pid is not running, and remove /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod/a28b2d553bc195a3/turbod.pid
  |     - check the logs at /Users/mpereira/Library/Application Support/turborepo/logs/a28b2d553bc195a3-frontend.log
  |     - the unix domain socket at /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod/a28b2d553bc195a3/turbod.sock has been removed
  | 
  |     You can also run without the daemon process by passing --no-daemon

 WARNING  failed to contact turbod. Continuing in standalone mode: connection to turbo daemon process failed.
    To quickly resolve the issue, try running:
    - $ turbo daemon clean

    To debug further - please ensure the following:
    - the process identified by the pid in the file at /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod/a28b2d553bc195a3/turbod.pid is not running, and remove /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod/a28b2d553bc195a3/turbod.pid
    - check the logs at /Users/mpereira/Library/Application Support/turborepo/logs/a28b2d553bc195a3-frontend.log
    - the unix domain socket at /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod/a28b2d553bc195a3/turbod.sock has been removed

    You can also run without the daemon process by passing --no-daemon

<rest of the output clipped>

It also seems that --no-daemon doesn't do anything for me:

$ turbo --no-daemon --cwd <turbo_directory> run dev --filter=<app>
 WARNING  failed to contact turbod. Continuing in standalone mode: connection to turbo daemon process failed.
    To quickly resolve the issue, try running:
    - $ turbo daemon clean

    To debug further - please ensure the following:
    - the process identified by the pid in the file at /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod/a28b2d553bc195a3/turbod.pid is not running, and remove /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod/a28b2d553bc195a3/turbod.pid
    - check the logs at /Users/mpereira/Library/Application Support/turborepo/logs/a28b2d553bc195a3-frontend.log
    - the unix domain socket at /var/folders/nn/120cjkpx75b8z_jcvjm04_940000gn/T/turbod/a28b2d553bc195a3/turbod.sock has been removed

    You can also run without the daemon process by passing --no-daemon

<rest of the output clipped>

[anthonyshew]

There are likely many daemons running on your development machine. The only way Turborepo's is different is that you know about it.

We're working towards our daemon becoming properly "invisible" like most other daemons. Please open an issue if your daemon is being problematic.

[aristofun]

There are likely many daemons running on your development machine. The only way Turborepo's is different is that you know about it.

Sorry, but this is a terrible justification. And a terrible solution.

You basically saying - we do it because everyone is doing it and we gonna make it even more hidden for you.

Realworld case — once we've spent the whole working day trying to resolve weird critical bug in our system. Until we found that some background daemon (hidden from us) silently messed up our files.

You cater to professional user, you should make things explicit rather than implicit.

[anthonyshew]

@aristofun, my explanation was moreso context for folks who aren't aware of the various background operations that are running on their machines. You're correct that the goal is ensure that the daemon is stable so that folks aren't bothered by errors coming from it, similar to the many other daemons that help developers every day.

Realworld case — once we've spent the whole working day trying to resolve weird critical bug in our system. Until we found that some background daemon (hidden from us) silently messed up our files.

When you say "real world case", do you mean that this is something that happened with the Turborepo daemon? If so, can you open an issue to describe the problem so we can address it? We definitely had some instability in the past around the daemon and are now under the impression that we've resolved those issues.

Also, we're working towards auditing the flags, environment variables, and turbo.json configuration options that exist in Turborepo and this sounds like one that we can put on the list to help out the folks interested here. In light of that, I'll open this issue back up so we have better visibility for this configuration. 👍

[hmnd]

I've been experiencing some sort of leak in the daemon causing it to peg a CPU core at 100% after some builds. I haven't been able to track down why this may be happening so not sure if it's worth opening an issue for that, but in the meantime I came up with this to always run without the daemon:

function turbo() { command turbo "$@" --no-daemon; }

(add to your ~/.bashrc or ~/.zshrc)

[jjhuff]

I've been experiencing some sort of leak in the daemon causing it to peg a CPU core at 100% after some builds. I haven't been able to track down why this may be happening so not sure if it's worth opening an issue for that, but in the meantime I came up with this to always run without the daemon:

function turbo() { command turbo "$@" --no-daemon; }

(add to your ~/.bashrc or ~/.zshrc)

Just hit this as well. Looks like it's looping on calling yarn --version.

[hmnd]

@jjhuff interesting... I'm using turbo with pnpm

[NicholasLYang]

Hi @jjhuff, could you send over the logs from your daemon? You should be able to see them by running turbo daemon logs or by going to the file listed in turbo daemon status. If you want, you can also send it to me at nicholas.yang@vercel.com

[wjoel]

I've seen it several times, I killed a 100% CPU turbo daemon just an hour ago. That's with pnpm. Will check the logs and the status next time it happens (seems to happen at least once a week).

[jjhuff]

@NicholasLYang This happens when I start it a second time. The first time, everything works great.

2024-04-29T16:29:53.223423Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:53.624224Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:54.034158Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:54.426060Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:54.808607Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:55.176551Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:55.542629Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:55.900112Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:56.266501Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:56.630643Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:57.000904Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:57.358331Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events
2024-04-29T16:29:57.731628Z  WARN turborepo_filewatch::package_watcher: lagged behind 1 processing file watching events

[lukpsaxo]

I can confirm - I also experience systematic problems with turbo.exe hanging in memory and blocking node_modules from being wiped out whenever I need it, not to mention redundant console messages about "standalone daemon not being found" or alike.

I have experienced significant speed up using the Daemon. With the daemon - under 1s - without the daemon - around 10 seconds.

So I wanted to get things set up seamlessly.. the problem I have experienced is with yarn v4 being unable to switch to a different version of turbo:

➤ YN0001: │ Error: EPERM: operation not permitted, unlink 'C:\git\SaxoTrader\node_modules\turbo-windows-64\bin\turbo.exe'
➤ YN0000: └ Completed in 0s 721ms
➤ YN0000: · Failed with errors in 7s 883ms

I couldn't find any documentation for turbo daemon but it seems turbo daemon clean stops the daemon. So I thought we should have a pre-install script that stops the daemon.

The trouble with yarn v4 is that pre-install runs after linking and there is no pre-link lifecycle hook (https://github.com/yarnpkg/berry/issues/5437). I even tried writing a plugin, but it seems yarn doesn't expose a way to make a plugin that works on install, pre linking and causes yarn to wait (I got validate workspace to kick off the command before linking, but then yarn doesn't wait :/). Maybe someone has a solution for this?

Another problem I got was that sometimes the turbo daemon clean would error - in my case the daemon was not running and this directory was already deleted, but I got this error:

Failed to remove log files: failed to remove directory `C:\git\xxx\.turbo\daemon`
Please remove manually: C:\git\xxx\.turbo\daemon
  x unable to complete daemon clean

so I also had to make the clean script ignore errors.

I also thought - lets start the daemon in postinstall.. but with both concurrently and npm-run-all it seems it would wait on yarn turbo daemon start and caused yarn to hang :/

Maybe this problem with locks on running processes only affects windows, but I really don't see how daemon is usable by people :/

[tilgovi]

I'd also like a way to opt out easily via config.

I do think it's true that many people run many daemons without being aware of it, but I think it's paternalistic to use that as a justification. I think many folks run background processes as a side effect of running an IDE, but the IDE can shut those down when it exits.

Take TypeScript as an example. Running tsc does not start a daemon. Running typescript-language-server does. An IDE user might be unaware of the latter, but it's still possible for the IDE to shut down the daemon when it exits. Something is managing the lifecycle.

What's missing and intrusive about the Turbo daemon is that it starts as a side effect of simply running turbo from the command line and never stops. It would absolutely be appropriate for the Turbo language server to act as a daemon for other Turbo commands to communicate with, but I think it's a questionable decision to have turbo run start it implicitly.

That said, it's actually fine if this is really what you think is the best user experience. It's certainly possible to pass --no-daemon. However, it's not possible today to run turbo watch --no-daemon.

So while I agree with voices above that think implicitly running the daemon by default is not a great choice, I can empathize with it and work around it. But my actual feature request would be to enable a turbo watch --no-daemon mode that runs the daemon code and the user interface code in the same process.

I'm a library maintainer and I'd like to use Turbo in my libraries, but I don't want to start daemons on contributor machines without their knowledge. If they choose to engage with Turbo via a language server for integration in their IDE that's great. But I'd like them to be able to run npm run test:watch and not end up with a daemon running without their knowledge.