Filewatcher ignore from `.gitignore` not working

ndom91 commented 4 months ago

Verify canary release

[X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

https://github.com/gitbutlerapp/gitbutler

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Linux

Which canary version will you have in your reproduction?

2.0.7-canary.1

Describe the Bug

A few times a day when trying to spin up my dev tasks (via turbo watch ..) it'll fail to start with the error message "fail to connect to daemon: server is unavailable: channel closed". After running daemon clear and/or daemon restart it usually works again for a while.

I finally noticed that turbo daemon logs will also tail the logs, so I ran this in the background while such an error occured. The error with turbo watch crashign always seems to occur while some Rust code is compiling and will log some output like this:

2024-07-16T12:24:41.025236Z ERROR turborepo_lib::daemon::server: package changes stream closed: channel closed
2024-07-16T12:24:54.905223Z ERROR turborepo_lib::daemon::server: package changes stream closed: channel closed
2024-07-16T12:24:57.837548Z  INFO turborepo_lib::daemon::server: triggering shutdown
2024-07-16T12:25:02.683079Z  WARN turborepo_lib::package_changes_watcher: changed_files: {AnchoredSystemPathBuf("crates/gitbutler-tauri/Cargo.toml")}
2024-07-16T12:25:02.683116Z  WARN turborepo_lib::package_changes_watcher: changed_packages: Ok(Some({WorkspacePackage { name: Root, path: AnchoredSystemPathBuf("") }}))
2024-07-16T12:25:09.210155Z  WARN turborepo_filewatch: encountered error watching filesystem enumerating recursive watch: IO error for operation on /opt/gitbutler/gitbutler/target/debug/build/atk-sys-9458f4fb2e56870d/rmetaahEQrj: No such file or directory (os error 2)
2024-07-16T12:25:09.231142Z ERROR turborepo_lib::daemon::server: package changes stream closed: channel closed
2024-07-16T12:25:40.004386Z ERROR turborepo_lib::daemon::server: package changes stream closed: channel closed
2024-07-16T12:25:45.043916Z ERROR turborepo_lib::daemon::server: package changes stream closed: channel closed
2024-07-16T12:25:48.147472Z  INFO turborepo_lib::daemon::server: triggering shutdown
2024-07-16T12:26:49.347045Z  INFO turborepo_lib::daemon::server: triggering shutdown

Notice the turborepo_filewatch warning line about no such file or directory

This is a Tauri project and the target/.. directory is the Rust compile output, so files are often being written and deleted quickly. This directory is already a part of our .gitignore. I also tried adding !target to the inputs array for the dev task. Unfortunately that didn't seem to ignore that directory either.

From what I understand, the directory should be automatically ignored by being in our .gitignore anyway. Am I missing something here? :thinking:

Expected Behavior

Turbo watcher doesn't crash on rust compiles.

To Reproduce

Clone https://github.com/gitbutlerapp/gitbutler
Install all pnpm dependencies
Ensure you have rust / Cargo setup
Run pnpm tauri dev in the root of the repository

Additional context

No response

NicholasLYang commented 4 months ago

Hi, thanks for the issue. We don't filter by .gitignore because the daemon needs to watch files that may be gitignored, such as your task outputs. We do this so we can detect if the task outputs haven't changed and therefore we don't need to do a cache restoration.

That said, clearly such an error shouldn't crash the package changes stream. I'll look into that and also see if we can add some gitignore filtering to the package changes watcher.

ndom91 commented 4 months ago

@NicholasLYang thanks for getting back to me! If .gitignore content isn't ignored by default, you may also want to reword some of the documentation here.

Also, can you confirm whether or not the inputs: ["!directory"] is supposed to work as I'm expecting? According to the docs you should be able to ignore certain files/directories from acting as inputs and being watched by the filewatcher. Is that right? I tried this as well, but it also didn't seem to work

vercel / turborepo