Polling based extension of the —watch-config flag that doesn’t rely on inotify/SIGHUP

[britton-from-notion]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

For context, I’m running vector on ECS. We build our initial configurations into the image however we’ve built a flow that allows deploying vector configuration via s3 -> lambda -> efs where vector listens for the reload. Allowing for configuration release to be independent from service deployment. The files are updated on the filesystem but EFS doesn’t support inotify so vector never notices that they changed or forced a reload.

Attempted Solutions

Currently our solution is to trigger a service restart via Lambda due to --watch-config not receiving inotify signals when leveraging an EFS based volume for vector configuration. This works but it is a lot of extra complexity that could be removed with direct support for --watch-config on EFS based volumes.

Proposal

It would be nice to be able to watch for file changes by polling the filesystem system for hash differences as well as the current support for inotify based updating.

References

Links

• Current polling implementation
• Notify crate's PollWatcher
• Original discord feature request message
• https://serverfault.com/questions/934397/inotify-does-not-work-on-efs-remote-filesystem
• https://stackoverflow.com/a/52439741

• 20989

Version

vector 0.40.0

[amribm]

I would like to work on this.

If I understand correctly: if --watch-config is only enabled we will use notify package's recommended watcher.

on the other side if --watch-config and --watch-config-interal is enabled vector will chose poll watcher from notify

@jszwedko @britton-from-notion pls confirm

[jszwedko]

Awesome, that'd be great @amribm . Thinking about this a bit more, I think it'd be better to make the flags a bit more explicit like:

• --watch-config enables the config watcher with default configuration
• --watch-config --watch-config-method=poll|inotify to configure which method to watch for new files (overriding default)
• --watch-config --watch-config-method=poll --watch-config-poll-interval-secs 30

In the past people have been understandable confused when flags have indirect behavior (e.g. enabling polling if --watch-config-internal is specified). I think the above flags are more self-evident, albeit verbose. I think this also aligns better with the spirit of https://github.com/vectordotdev/vector/blob/master/docs/specs/configuration.md#option-polymorphism, which discusses "polymorphic" config options such as this one.

What do you think?

[amribm]

great, this seems more explicit.

@jszwedko if user selected --watch-config-method=poll and didn't mention -watch-config-poll-interval-secs can we set the defalult interval seconds to 30?