dora-rs / dora

DORA (Dataflow-Oriented Robotic Architecture) is middleware designed to streamline and simplify the creation of AI-based robotic applications. It offers low latency, composable, and distributed dataflow capabilities. Applications are modeled as directed graphs, also referred to as pipelines.
https://dora-rs.ai
Apache License 2.0
1.5k stars 79 forks source link

Try to rewatch removed files in `--attach` #532

Open phil-opp opened 3 months ago

phil-opp commented 3 months ago

Vim creates a new file inode when modifying files, which results in a 'remove' event. This commit handles this by treating 'remove' events as a trigger for reloads. After sending the reload command, the watcher now tries to watch the same path again, which watches the newly created file.

Fixes #530

phil-opp commented 3 months ago

@hbuxiaofei Could you please try whether this PR fixes #530 for you?

hbuxiaofei commented 3 months ago

@hbuxiaofei Could you please try whether this PR fixes #530 for you?

Yes, this patch does work, but it needs to add the ModifyKind::Metadata and ModifyKind::Name events to work well.

EventKind::Modify(ModifyKind::Metadata(_)) => false,   
EventKind::Modify(ModifyKind::Name(_)) => true,        

Maybe it's because vim creates a temporary .xxx.swp file.

And there are some small problems, some warning logs will appear.

  2024-06-06T07:41:58.048671Z  WARN dora::attach: failed to watch `/home/rayylee/github/code-test/rust/dora-test/python-operator-dataflow/plot.py` again: No such file or directory (os error 2) -> further modifications will be ignored
    at binaries/cli/src/attach.rs:114
phil-opp commented 3 months ago

Thanks for testing!

And there are some small problems, some warning logs will appear.

Hmm, this warning indicates that the re-watching failed, so subsequent file changes will be ignored. So this is problematic. Do the warnings still occur after you added the ModifyKind::Metadata and ModifyKind::Name events?

hbuxiaofei commented 3 months ago

Thanks for testing!

And there are some small problems, some warning logs will appear.

Hmm, this warning indicates that the re-watching failed, so subsequent file changes will be ignored. So this is problematic. Do the warnings still occur after you added the ModifyKind::Metadata and ModifyKind::Name events?

After adding the ModifyKind::Metadata and ModifyKind::Name events, the warning will appear. But hot-reloading works good.

phil-opp commented 3 months ago

Could you post the relevant parts of your vim config. I'm trying to reproduce this, but I keep getting normal Modify events.

After adding the ModifyKind::Metadata and ModifyKind::Name events, the warning will appear. But hot-reloading works good.

Adding ModifyKind::Name seems reasonable, but I'm not sure about ModifyKind::Metadata. Ideally, I want to avoid any watch errors if possible. Or alternatively, we could retry a failed watch operation if there is a race between the creation of the new file and the Remove event of the old file.

hbuxiaofei commented 3 months ago

Could you post the relevant parts of your vim config. I'm trying to reproduce this, but I keep getting normal Modify events.

After adding the ModifyKind::Metadata and ModifyKind::Name events, the warning will appear. But hot-reloading works good.

Adding ModifyKind::Name seems reasonable, but I'm not sure about ModifyKind::Metadata. Ideally, I want to avoid any watch errors if possible. Or alternatively, we could retry a failed watch operation if there is a race between the creation of the new file and the Remove event of the old file.

Here is my vim configuration repository: https://github.com/hbuxiaofei/vimll

But I don't think it has anything to do with the configuration. I cleared my vim configuration and still received this events:

>>> recv events: Event { kind: Modify(Metadata(Any)), paths: ["/home/rayylee/github/code-test/rust/dora-test/python-operator-dataflow/plot.py"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }
>>> recv events: Event { kind: Remove(File), paths: ["/home/rayylee/github/code-test/rust/dora-test/python-operator-dataflow/plot.py"], attr:tracker: None, attr:flag: None, attr:info: None, attr:source: None }

Vim version :

$ vim --version
VIM - Vi IMproved 9.1 (2024 Jan 02, compiled Mar 31 2024 00:15:53)