Watchman fails with 'Operation not permitted'

[orome]

Since updating yesterday using Hombrew on macOS, watchman always fails with the error:

{
    "version": "2022.05.30.00",
    "error": "std::__1::system_error: open: /Path/To/Project: Operation not permitted"
}

Does the 2022.05.30.00 version of Watchman work? if so, why would it fail (for me at least) suddenly after being updated?

As far as I know the only difference between the configurations of my machine today and yesterday (when Watchman worked fine) is that I ran brew upgrade watchman. I have confirmed all the necessary file permissions are in place and have rebooted, reinstalled, terminated, etc.

UPDATE: A solution to get things working while we await a fix to Watchman is provided below:

watchman shutdown-server       # To ensure an old version isn't still running
brew uninstall watchman        # Get rid of the buggy (e.g., 2022.05.30.00) version
curl https://raw.githubusercontent.com/Homebrew/homebrew-core/8651d8e23d308e564414188509f864e40548f514/Formula/watchman.rb > "$(brew --repository)/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb"
brew install watchman          # This installs the functioning 2022.05.16.00 version dowloaded above
brew pin watchman              # Tell brew to leave this older version alone (don't forget to unpin once this problem is solved)
cd "$(brew --repository)/Library/Taps/homebrew/homebrew-core/Formula/"
git checkout --  watchman.rb   # To avoid brew showing warnings about having uncommitted git changes

Note that for those who have found other solutions, you are not experiencing the same issue, but of you are doing things like granting watchman full disk access, you may be reducing your security unnecesarily.

Also, if you are on macOS, and the above doesn't work on its own, it may pay off to do the following and try the above again:

sudo tccutil reset SystemPolicyDocumentsFolder
sudo tccutil reset SystemPolicyAllFiles

If you do this, you will need to again grant access to those apps that need it (they will ask).

UPDATE:As of 2022.06.27.00 Watchman appears to work again.

If you followed the instructions above, just

brew unpin watchman

and update it as usual. Then (and from then on), to be sure things work as expected, after every update of Watchman:

watchman watch-del-all
watchman shutdown-server 

[orome]

Still stuck unable to do any development because of Watchman. I've confirmed that all I did was

$ brew update
$ brew install watchman

This was enough to transform a working stack into one that always fails as above for all directories. Is there a way to install a previous version using brew install (not ZIP, see below).

(I tired installing from ZIP but this fails too due .lib load and security issues (#1031.)

[orome]

In an attempt to fix this I've run

brew update
brew uninstall watchman
sudo tccutil reset SystemPolicyDesktopFolder
sudo tccutil reset SystemPolicyDeveloperFiles
sudo tccutil reset SystemPolicyDocumentsFolder
sudo tccutil reset SystemPolicyDownloadsFolder
sudo tccutil reset SystemPolicyNetworkVolumes
sudo tccutil reset SystemPolicyRemovableVolumes
sudo tccutil reset SystemPolicySysAdminFiles

then rebooted and run

brew install watchman
watchman

That results in a series of library errors that are cleared by reinstalling several of watchman's dependencies. Once that's done I did

brew update
brew uninstall watchman

then rebooted again and did

brew install watchman
watchman

and still get the same error:

{
    "version": "2022.05.30.00",
    "error": "std::__1::system_error: open: /Path/To/Project: Operation not permitted"
}

I then gave watchman Full Disk Access, but continue to see the same error.

Also at no point during this process does launching watchman result in the expected request for access to the Documents folder (as all other apps that need it do correctly following the tccutil resets).

[beamcode]

same issue damn

[orome]

@beamcode Really vexing bug. I basically redid all my permissions in a failed attempt to get rid of it.

It seems to be part of a longer running problem with watchman: I had multiple (about 8) entries for watchman in my Files & Folders settings, all for executable versions that were no longer present (as indicated by the lack of an icon) and that were causing issues with getting settings to stick.

So there's some kind of bad behavior by watchman relating to File & Folders permissions — and possibly how it identifies itself to macOS — that has been going on for a while, it appears.

[beamcode]

https://i.ibb.co/8X5N1tL/Screenshot-2022-06-01-at-23-05-04.png i manually added watchman to the full disk access what else can i do i reinstall like 5 times Maybe its something to do with a recent macos version because i just got my new m1 mac like 3 days ago and updated everything to latest updates idk

[orome]

@beamcode I manually added Full Disk Access but that didn't make a difference.

FWIW I'm running on an old config: MacBook Pro (Retina, 13-inch, Mid 2014); macOS 11.6.6.

[calluthom]

The only thing that seems to work is moving your entire project to your Home folder. Nothing else seems to work. It seems to be affecting a lot of other users right now, likely a bug with some new macos security update with file access permissions.

I have the same error after following the React Native getting started guide, on both xcode and expo cli.

I hope the watchman team looks into this!

[orome]

I have the same error after following the React Native getting started guide, on both xcode and expo cli.

Yes, this bug kills expo start: no development is happening here.

[woyounai]

guys,you need reinstall "version": "2022.05.16.00".

[orome]

guys,you need reinstall "version": "2022.05.16.00".

I tried that both using the ZIP (which doesn't work at all #1031) and using previous Hombrew formulas, none of which produced a functioning installation. How did you manage to install a previous version?

[orome]

The only thing that seems to work is moving your entire project to your Home folder.

Confirming that this is also the only thing that works for me.

Not a practical solution, but it will allow some headway to be made.

[gorhom]

same issue here, it only worked when moving the project to the home folder 😕

MacOS 12.4 ( M1 )

[oetjen]

This is definetly a problem with version 2022.05.30.00. If you rollback to 2022.05.16.00, it works again. Howto: See https://remarkablemark.org/blog/2017/02/03/install-brew-package-version/

TLDR:

brew uninstall watchman
curl https://raw.githubusercontent.com/Homebrew/homebrew-core/8651d8e23d308e564414188509f864e40548f514/Formula/watchman.rb > /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb
brew install watchman
brew pin watchman # Don't forget to unpin once this problem is solved...
cd /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/
git checkout --  watchman.rb

[gorhom]

This is definetly a problem with version 2022.05.30.00. If you rollback to 2022.05.16.00, it works again. Howto: See https://remarkablemark.org/blog/2017/02/03/install-brew-package-version/

TLDR:

brew uninstall watchman
curl https://raw.githubusercontent.com/Homebrew/homebrew-core/8651d8e23d308e564414188509f864e40548f514/Formula/watchman.rb > /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb
brew install watchman
brew pin watchman # Don't forget to unpin once this problem is solved...
git checkout --  /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb

thanks @oetjen! your solution did fix the issue 🙌

[orome]

@oetjen I get:

/opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb: No such file or directory

I don't even have an /opt/homebrew/, just an /opt/homebrew-cask/, which is empty.

I do have a

/usr/local/Homebrew/Library/Taps

which contains a caskroom and a homebrew. Should I be using

/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb

instead?

[shoopi12]

2022.05.30.00 appears to be broken.

Downgrading to a previous version is surprisingly not easy.

Decided to save some time and move move it to a different folder and it worked. Will bring it back once the bug is fixed I suppose.

[orome]

Downgrading to a previous version is surprisingly not easy.

@shoopi12 I recommend giving it a shot. The instructions above work. The only things to be careful of are:

1. If you are on an Intel Mac use

   /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/

   for the Formula path.

2. Be sure to cd to that directory before the git checkout step.
3. Run watchman shutdown-server afterwards to make sure an older version isn't lingering around.

[shoopi12]

@orome tried that, but for some reason I still get the error with the higher version:

while watchman -v shows the lower version:

What I did:

brew uninstall watchman

curl https://raw.githubusercontent.com/Homebrew/homebrew-core/8651d8e23d308e564414188509f864e40548f514/Formula/watchman.rb > /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb

*edit*: brew install watchman

brew pin watchman

cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula

git checkout

[orome]

@orome tried that, but for some reason I still get the error with the higher version...`

@shoopi12 You omitted brew install watchman!

[shoopi12]

@orome sorry I omitted it from my Comment, I did the step of course

Edit: After restarting my computer it worked. I'm summing up the steps here once again for anyone else:

brew uninstall watchman

curl https://raw.githubusercontent.com/Homebrew/homebrew-core/8651d8e23d308e564414188509f864e40548f514/Formula/watchman.rb > /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb

brew install watchman

brew pin watchman

cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula

git checkout

- restart macOS

If needed, give watchman Files and Folders / Full Disk Access permission, maybe uncheck and recheck it. When I tried npm start again I got the blessed alert "watchman would like to access your Files and Folders". After confirming everything worked.

[liquidvisual]

Finally got this to work. I'm on an Intel iMac running Monterey. Here were my steps:

brew uninstall watchman

curl https://raw.githubusercontent.com/Homebrew/homebrew-core/8651d8e23d308e564414188509f864e40548f514/Formula/watchman.rb > /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb

brew install watchman

brew pin watchman

cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/

git checkout --  watchman.rb

This did nothing initially, but then I ran:

sudo npm run test as discussed here.

A permission window then popped up where I was finally able to give access to my Desktop (project is living on my Desktop).

I then rebooted the terminal - didn't need to restart macOS (although it couldn't hurt). I also didn't touch anything in "Security & Privacy" under System Preferences like some have mentioned.

I ran npm run start and it worked, finally. My god, that was a fun 8 hours.

[orome]

I see 2022.06.06.00 is out but this is still open. I that correct?

Does anyone who has been having this issue have experience with 2022.06.06.00?

[StorkZ]

I have just installed 2022.06.06.00 and the same problem still persists. Currently I am trying to get it to work using one of the solutions above.

[khaled-scribd]

brew uninstall watchman curl https://raw.githubusercontent.com/Homebrew/homebrew-core/8651d8e23d308e564414188509f864e40548f514/Formula/watchman.rb > /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb brew install watchman brew pin watchman # Don't forget to unpin once this problem is solved... cd /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/ git checkout -- watchman.rb

Can someone explain why the last step git checkout -- watchman.rb is required?

[neoswallow]

Can someone explain why the last step git checkout -- watchman.rb is required?

It is to reset the change you just made by replacing watchman.rb with a previous version. Brew update will break if you have un-committed changes.

[zhangheng0317]

I use 2022.06.06.00, its OK

[l0gicgate]

@zhangheng0317 2022.06.06.00 doesn't work for me. Same bug still.

[cktaylor]

Is this potentially caused by this change: https://github.com/facebook/watchman/commit/db90c1bfc87c76d5169ed4958b778efa2e82c188 ?

Seemingly on mac watchman is now launched via /bin/sh under an assumption that watchman would inherit Full Disk Access from /bin/sh. But either /bin/sh isn't guaranteed to have such access in all cases or it's not inherited by watchman as intended.

Furthermore - by launching in this way - I believe it may mean the OS doesn't prompt the user for File/Folder access like it otherwise was in past versions of watchman.

The timing certainly lines up with this change in question first being built into 2022.05.30

[orome]

I use 2022.06.06.00, its OK

@zhangheng0317 Were you having the issue described in the OP (with 2022.05.30)?

[orome]

Furthermore - by launching in this way - I believe it may means the OS doesn't prompt the user for File/Folder access like it otherwise was in past versions of watchdog.

@cktaylor Correct. 2022.05.30 never asks for File & Folder permission.

[zhangheng0317]

I have the same bug in 2022.05.30, so i reinstall all and upgrade watchman

[TwanLuttik]

I can also confirm this is an issue after a clean factory reset on my macbook m1 max with watchman version 2022.06.06.00.

[DaniloPootaren]

I can confirm also that issue still persists with M1 chips. I've tried all the solutions given above and I could not resolve it.

[neoswallow]

The workaround worked around initially but it's broken again and now I'm getting different error.

app/node_modules/metro-hermes-compiler/src/emhermesc.js:77
          throw ex;
          ^

Error: EMFILE: too many open files, watch
    at FSEvent.FSWatcher._handle.onchange (node:internal/fs/watchers:204:21)
Emitted 'error' event on NodeWatcher instance at:
    at NodeWatcher.checkedEmitError (app/node_modules/sane/src/node_watcher.js:143:12)
    at FSWatcher.emit (node:events:527:28)
    at FSEvent.FSWatcher._handle.onchange (node:internal/fs/watchers:210:12) {
  errno: -24,
  syscall: 'watch',
  code: 'EMFILE',
  filename: null
}
error Command failed with exit code 7.

And I also get this error trying to run any watchman command directly:

watchman shutdown-server                                                                                               
dyld[82779]: Symbol not found: __ZN5folly19shared_mutex_detail25getMaxDeferredReadersSlowERNSt3__16atomicIjEE
  Referenced from: /opt/homebrew/Cellar/watchman/2022.05.16.00/bin/watchman
  Expected in: /opt/homebrew/Cellar/folly/2022.06.06.00/lib/libfolly.0.58.0-dev.dylib
[1]    82779 abort      watchman shutdown-server

And I did run brew upgrade multiple times since the workaround steps. Maybe we need pin more than just watchman itself.

[neoswallow]

It is possible I had folly@2022.05.30.00 when I reverted watchman to 2022.05.16.00. I was going to revert folly to previous version but it's depend of other packages and I didn't want to break other packages to fix watchman. Redoing workaround steps didn't help. So it is possible this workaround steps may not help anyone else who tried after folly (or fb303 or any other watchman dependencies) got updated to a version that isn't compatible with watchman@2022.05.16.00. I had no option but to move the repo from external drive to somewhere in home directory after which metrobundler is now working again.

[kieuquang1110tt]

MacOs Monterey 12.4 Still got this problem. Watchman 2022.06.06.00. Pls help.

[Mixpeal]

MacOs Monterey 12.4 Still got this problem. Watchman 2022.06.06.00. Pls help.

It's weird, but just

brew uninstall watchman

worked for me.

[orome]

FWIW, things have been working consistently for me with pinned watchman 2022.05.16.00 (only) since following the steps above, with no other packages or dependencies pinned.

[kbeliauski]

Does anyone know what is the problem? I keep getting this:

Error: EMFILE: too many open files, watch at FSEvent.FSWatcher._handle.onchange (internal/fs/watchers.js:178:28)

Thank you!

I downgraded to the 2022.05.16.00, and the error with permission is gone. Now I keep getting this.

[robert-go]

This is definetly a problem with version 2022.05.30.00. If you rollback to 2022.05.16.00, it works again. Howto: See https://remarkablemark.org/blog/2017/02/03/install-brew-package-version/

TLDR:

brew uninstall watchman
curl https://raw.githubusercontent.com/Homebrew/homebrew-core/8651d8e23d308e564414188509f864e40548f514/Formula/watchman.rb > /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/watchman.rb
brew install watchman
brew pin watchman # Don't forget to unpin once this problem is solved...
cd /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/
git checkout --  watchman.rb

➜ ~ watchman -v dyld[3623]: Symbol not found: (__ZN5folly19shared_mutex_detail25getMaxDeferredReadersSlowERNSt3__16atomicIjEE) Referenced from: '/opt/homebrew/Cellar/watchman/2022.05.16.00/bin/watchman' Expected in: '/opt/homebrew/Cellar/folly/2022.06.06.00/lib/libfolly.0.58.0-dev.dylib' [1] 3623 abort watchman -v

[robert-go]

it's not working for me.

[legion-zver]

@robert-go try adding "sh" permissions to the disk

[TwanLuttik]

@robert-go try adding "sh" permissions to the disk

What is the file location for sh? (I'm on mac btw)

[musasoftlabx]

This fix will work. Kindly try it

Install latest watchman version (brew install watchman)

1. Go to System preferences > Security & Privacy then scroll down
2. Select Full Disk Access
3. Unlock it and click + button to add new app/tool
4. Select Macintosh HD in the folders list. (For new users, if u cant see this folder, fo to finder preferences > sidebar and check it.)
5. Press this combination shift Command . (don't forget the dot) at the same time to see all the directories list
6. Click opt > homebrew > bin (Tested only on MacOS Monterey. If using another version, search watchman under Macintosh HD).
7. Look for watchman in the list then select it.
8. Once added, run ur program again.

Make sure ur project folder is under /Users//. Don't put it under documents or any other folder there. It will refuse due to some weird restrictions.

[Seunope]

brew uninstall watchman

Then I ran the program again. In my case npm run android. This resolved the issue.

[chadaustin]

I suspect this regression was introduced by https://github.com/facebook/watchman/commit/db90c1bfc87c76d5169ed4958b778efa2e82c188

We're working on a more targeted fix to the original issue, but in the meantime hope to revert the offending change.

Until then, @musasoftlabx's fix that grants Watchman Full Disk Access is worth trying.

[legion-zver]

@robert-go try adding "sh" permissions to the disk

What is the file location for sh? (I'm on mac btw)

which sh

/bin/sh

[nathanaelphilip]

@chadaustin any updates?

[Ebeldev]

@musasoftlabx

What are the weird restrictions, you are talking about?

I did follow all the steps you mentionned.

All my react-native projects are under the documents folder. Never got any problem since 5 years.

[musasoftlabx]

@Ebeldev , tested mine with M1 Pro with Monterey and got errors when I placed my project under /Documents. I think previous versions have no issue. Am guessing the OS restricts those folders unless you chmod to change permissions