search

envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.99k stars 4.81k forks source link

inotify_fd_ assertion is misleading #12383

Open mook-as opened 4 years ago

mook-as commented 4 years ago

Description:

9098 added a description on the assertion for when inotify_fd_ failed to initialize. The assertion says to Consider increasing value of user.max_inotify_watches via sysctl; however, it's asserting that inotify_init1(IN_NONBLOCK) succeeds, which creates new inotify instances instead of watches. So the user should be increasing user.max_inotify_instances instead.

Repro steps:

dio commented 4 years ago

cc. @sriduth

sriduth commented 4 years ago

The assertion message is misleading - I will open a PR to fix this, will also try to see if we can use errno to deliver a better error message.

sriduth commented 4 years ago

I made a patch here: https://github.com/sriduth/envoy/commit/acf3831b4b2a4d636db0033c955e30f74a9ea6d5 but based on the original issue reported #7130, it looks like inotify_init1 can fail of either user.max_inotify_watches is set to a low value - ill dig deeper into this.

Also, we could improve the message logged if inotify_add_watch fails in the WatcherImpl::addWatch method, currently it just prints No space left on device which is std::strerror I think.

sriduth commented 4 years ago

in initial report for #7130 sysctl -w user.max_inotify_watches=524288 was mentioned as the fix used

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

github-actions[bot] commented 3 years ago

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

Swaagie commented 6 months ago

@sriduth this is still an issue. I recently ran into this and can confirm using max_inotify_instances resolved it rather than using max_inotify_watches. Do you intent to PR the changes from your fork?

edit: these changes https://github.com/sriduth/envoy/commit/acf3831b4b2a4d636db0033c955e30f74a9ea6d5

github-actions[bot] commented 2 days ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.