Open oscarotero opened 1 year ago
I can't reproduce, please provide a minimal example with reproduction steps
tested with
a.ts
const watcher = Deno.watchFs(".");
for await (const event of watcher) {
console.log(event);
event.paths.forEach((path) => {
Deno.copyFileSync(path, "/dev/null"); // without this line I get the same events
});
}
terminal 1 run: deno run -A a.ts
terminal 2 run: touch hello
result
[Object: null prototype] {
kind: "create",
paths: [ "/home/mrcool/dev/deno/lab/tmp/XRm/./hello" ],
flag: null
}
[Object: null prototype] {
kind: "modify",
paths: [ "/home/mrcool/dev/deno/lab/tmp/XRm/./hello" ],
flag: null
}
[Object: null prototype] {
kind: "access",
paths: [ "/home/mrcool/dev/deno/lab/tmp/XRm/./hello" ],
flag: null
}
I have reproduced it in this zip file:
deno run -A main.ts
My guess is this is due to the large file size, and maybe the os copying it in chunks and each chunk triggers a modify event (its not infinite loop, its just a lot of events)
I imagine you get the same things without deno, by using inotify directly in linux
I tried inotify directly (with C) and its the same number of events, so it seems this is just how it works
Just clarify my OS is no Linux but macOS, and in my case it never stops (or at least it takes many seconds).
you can try with this file (courtesy of chatgpt)
a.c
clang a.c
./a.out
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/inotify.h>
#define EVENT_SIZE (sizeof(struct inotify_event))
#define BUF_LEN (1024 * (EVENT_SIZE + 16))
int main() {
int fd, wd, length;
char buffer[BUF_LEN];
fd = inotify_init();
if (fd < 0) {
perror("inotify_init");
exit(EXIT_FAILURE);
}
wd = inotify_add_watch(fd, "./src", IN_ALL_EVENTS);
if (wd < 0) {
perror("inotify_add_watch");
exit(EXIT_FAILURE);
}
printf("Watching current directory for events...\n");
while (1) {
length = read(fd, buffer, BUF_LEN);
if (length < 0) {
perror("read");
exit(EXIT_FAILURE);
}
int i = 0;
while (i < length) {
struct inotify_event *event = (struct inotify_event *) &buffer[i];
if (event->len) {
if (event->mask & IN_CREATE)
printf("File/directory created: %s\n", event->name);
if (event->mask & IN_DELETE)
printf("File/directory deleted: %s\n", event->name);
if (event->mask & IN_MODIFY)
printf("File modified: %s\n", event->name);
if (event->mask & IN_ATTRIB)
printf("File attributes modified: %s\n", event->name);
if (event->mask & IN_MOVE)
printf("File/directory moved: %s\n", event->name);
}
i += EVENT_SIZE + event->len;
}
}
inotify_rm_watch(fd, wd);
close(fd);
return 0;
}
oh actually mac doesn't use inotify probably
I can't test this but here is what chatgpt says for macos (you can alwyas fix the code if needed)
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/event.h>
int main() {
int kq, dirfd, num_events, i;
struct kevent event;
struct timespec timeout = {0, 0};
// Open the current directory
dirfd = open("./src", O_RDONLY);
if (dirfd == -1) {
perror("open");
exit(EXIT_FAILURE);
}
// Create a new kernel event queue
kq = kqueue();
if (kq == -1) {
perror("kqueue");
exit(EXIT_FAILURE);
}
// Register a new event in the event queue
EV_SET(&event, dirfd, EVFILT_VNODE, EV_ADD | EV_ENABLE | EV_CLEAR, NOTE_WRITE | NOTE_DELETE | NOTE_RENAME, 0, NULL);
// Attach the event to the event queue
if (kevent(kq, &event, 1, NULL, 0, &timeout) == -1) {
perror("kevent");
exit(EXIT_FAILURE);
}
printf("Watching current directory for events...\n");
while (1) {
// Wait for events to occur
num_events = kevent(kq, NULL, 0, &event, 1, &timeout);
if (num_events == -1) {
perror("kevent");
exit(EXIT_FAILURE);
} else if (num_events > 0) {
// Process the event
if (event.fflags & NOTE_WRITE)
printf("File modified: %s\n", event.udata);
if (event.fflags & NOTE_DELETE)
printf("File/directory deleted: %s\n", event.udata);
if (event.fflags & NOTE_RENAME)
printf("File/directory renamed: %s\n", event.udata);
}
}
close(kq);
close(dirfd);
return 0;
}
Thanks, but I'm not familiarized with C code.
I don't know if the bug is in the OS or Deno, but I think Deno should fix these issues and try to work the same in all operating systems. The way it works now is: on modify a file, copy it to other folder, which modify the file again, so copy the file again...
According to the documentation, the available event types are "any", "access", "create", "modify", "remove" and "other". Copying a file shouldn't modify the original file, but only access to it. This would allow to filter out the "access" events in the watcher and break the infinite loop.
But it not modifying the original file, it's modifying the file being created (by writing to it in chunks)
Just to make sure , is what you get an infinite loop or a lot of events (as in does the logging stops after a while or doesn't?)
In this video, you can see what happens in my computer.
https://github.com/denoland/deno/assets/377873/c791a202-9eb2-42b4-bc4a-8b03a9a9cdc0
Seems to be a bug in the mac implementation @dsherret maybe you can give it a look (or at least I think issue should be labeled)
to reproduce
wget https://github.com/denoland/deno/files/11691835/watcher.zip
unzip watcher.zip
cd watcher
deno run -A main.ts
In another terminal
cp dest/neom-bhKqZNZeAR0-unsplash.jpg src/neom-bhKqZNZeAR0-unsplash.jpg
Now the terminal show an infinite loop with modify events, doesn't happen on linux, but happens on macos
seems like Deno.watch is just a simple wrapper over notify crate https://github.com/denoland/deno/blob/e1be2bb80d2c9e7f56221d53349cdcea6d76ddde/runtime/ops/fs_events.rs#L96
which had deadlocks with macos before https://github.com/notify-rs/notify/issues/118
@oscarotero maybe you can give a shot to upgrading to the latest stable version https://github.com/denoland/deno/blob/main/Cargo.toml#L109 changing it to 6
@sigmaSd Thanks for your time researching on this issue. I'm glad to help but I don't know how do that (I'm not an engineer).
I have the following code:
Every time a file is modified, the watcher detects the change, so the file is copied to the destination folder. But seems like
Deno.copyFileSync
somehow triggers a new "modify" event to the original file in the watcher, so it's an infinite loop.As a workaround, I can read and write the file content
I think this is a bug, because
copyFile
shouldn't modify the original file.My Deno version: