Enable "fileHasher" for MacOS FileWatcher

[artem-barmin]

Trigger file changes only when content of file is changed.

Reason to suggest: my current setup with shadow-cljs on MacOS triggers compile multiple times after hitting "Cmd+S". That's definitely not an editor issue because I replicated this behaviuor even with simple "touch file.cljs" from terminal.

With suggested change watch triggered only in case when file content really changed, thus omitting false positive triggers

[thheller]

Which macOS version does this happen with? Apple Hardware? Any specific Filesystem or default?

[artem-barmin]

artem@MacBook-Air-3 ~ % system_profiler SPSoftwareDataType SPHardwareDataType
Software:

    System Software Overview:

      System Version: macOS 14.0 (23A344)
      Kernel Version: Darwin 23.0.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: MacBook Air (4)
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 6 days, 1 hour, 34 minutes

Hardware:

    Hardware Overview:

      Model Name: MacBook Air
      Model Identifier: Mac14,2
      Model Number: MLY43ZE/A
      Chip: Apple M2
      Total Number of Cores: 8 (4 performance and 4 efficiency)
      Memory: 8 GB
      System Firmware Version: 10151.1.1
      OS Loader Version: 10151.1.1
      Activation Lock Status: Enabled

artem@MacBook-Air-3 ~ % diskutil list
/dev/disk0 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *500.3 GB   disk0
   1:             Apple_APFS_ISC Container disk1         524.3 MB   disk0s1
   2:                 Apple_APFS Container disk3         494.4 GB   disk0s2
   3:        Apple_APFS_Recovery Container disk2         5.4 GB     disk0s3

/dev/disk3 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +494.4 GB   disk3
                                 Physical Store disk0s2
   1:                APFS Volume Macintosh HD            9.8 GB     disk3s1
   2:              APFS Snapshot com.apple.os.update-... 9.8 GB     disk3s1s1
   3:                APFS Volume Preboot                 5.6 GB     disk3s2
   4:                APFS Volume Recovery                888.4 MB   disk3s3
   5:                APFS Volume Data                    201.2 GB   disk3s5
   6:                APFS Volume VM                      5.4 GB     disk3s6

[artem-barmin]

Basically everything is super default. No specific tweaks to system was done

[thheller]

Unfortunately I don't have an M2 (or any M) mac available, so I cannot reproduce this and would like independent confirmation first.

The trouble with the file hashing is that touch some.cljs is no longer possible. I know that some people use that to trigger builds that rely on side-effecting/external macros or so. I don't want to break that.

Also there have been previous reports on this where the source of the multiple compile triggers was outside of their editor but from some other tools that watched the filesystem in similar ways and triggered a file touch. Can't remember what that was exactly though, maybe Spotlight? Would help to figure out what actually causes this behavior before working arround it.

[artem-barmin]

Yes, probably you are right about other software. However I don't know how to check what exactly is triggering update. Sometimes one "Save" triggers 4-5 compile operations one after another.

If I add new config flag to control this behaviour, like: :watch-mode :hash will you accept pull request?

[thheller]

First I'd like to gather some more data on what is actually happening.

Can you try npx shadow-cljs clj-repl and then

(def x (shadow.cljs.devtools.server.fs-watch/start {} [(clojure.java.io/file ".")] ["edn"] prn))

This will watch all .edn files in the project root directory. So you can do a touch shadow-cljs.edn and it'll just print all the fs events it gets. I get this

[{:dir #object[java.io.File 0x2a135874 "."], :name "shadow-cljs.edn", :ext "edn", :file #object[java.io.File 0x24d4a346 ".\\shadow-cljs.edn"], :event :mod}]

[artem-barmin]

Of course. I ran command, then went to feed my cat I got this in console(without any explicit file modifications made by me):

shadow.user=> [{:dir #object[java.io.File 0x2b44c6e "."], :name "node_modules/zprint-clj/deps.edn", :ext "edn", :file #object[java.io.File 0x2683b8a9 "./node_modules/zprint-clj/deps.edn"], :event :mod}]
[{:dir #object[java.io.File 0x2b44c6e "."], :name "node_modules/zprint-clj/deps.edn", :ext "edn", :file #object[java.io.File 0x4c0230a0 "./node_modules/zprint-clj/deps.edn"], :event :mod}]
[{:dir #object[java.io.File 0x2b44c6e "."], :name ".clj-kondo/rewrite-clj/rewrite-clj/config.edn", :ext "edn", :file #object[java.io.File 0x3947bb9e "./.clj-kondo/rewrite-clj/rewrite-clj/config.edn"], :event :mod}]
[{:dir #object[java.io.File 0x2b44c6e "."], :name "node_modules/shadow-cljs/cli/default-config.edn", :ext "edn", :file #object[java.io.File 0x19b047e4 "./node_modules/shadow-cljs/cli/default-config.edn"], :event :mod}]
[{:dir #object[java.io.File 0x2b44c6e "."], :name ".clj-kondo/rewrite-clj/rewrite-clj/config.edn", :ext "edn", :file #object[java.io.File 0x22cde28b "./.clj-kondo/rewrite-clj/rewrite-clj/config.edn"], :event :mod}]
[{:dir #object[java.io.File 0x2b44c6e "."], :name "public/js/manifest.edn", :ext "edn", :file #object[java.io.File 0x7f974142 "./public/js/manifest.edn"], :event :mod}]
[{:dir #object[java.io.File 0x2b44c6e "."], :name ".shadow-cljs/classpath.edn", :ext "edn", :file #object[java.io.File 0x7b889e65 "./.shadow-cljs/classpath.edn"], :event :mod}]
[{:dir #object[java.io.File 0x2b44c6e "."], :name ".clj-kondo/rewrite-clj/rewrite-clj/config.edn", :ext "edn", :file #object[java.io.File 0x601c6997 "./.clj-kondo/rewrite-clj/rewrite-clj/config.edn"], :event :mod}]

And that's not an end. Events keep arriving

[artem-barmin]

I found root problem: my project was located on the iCloud drive and Finder process was responsible for touching files. Moved folder to regular drive and problem is (probably) solved. I will try to watch how things are going during day and will write results

[thheller]

I wonder what modifications this does, probably just metadata things. Wonder if those could be detected and filtered out?

[artem-barmin]

Yes, that's definitely metadata like this: https://developer.apple.com/library/archive/documentation/CoreServices/Reference/MetadataAttributesRef/Articles/iCloudAttrs.html

From quick look in file watcher code in library that you use - not possible. That's why they implemented hasher to compare content

[thheller]

Is this solved now or do you still want an option for hashing? I don't think I want it as a default, but an option is fine.

[artem-barmin]

It's fully fixed.

Personally for me option is not needed now, but it would be great to cover this issue in Troubleshooting section of documentation.

During my investigation I tried to search for similar problems in Issues and docs, but without any luck. Ended up trying to fix it with file hasher by my own. If iCloud+file watcher issue was covered - everything would be much easier