VOLTTRON / volttron

VOLTTRON Distributed Control System Platform

https://volttron.readthedocs.io/

Other

452 stars 217 forks source link

Fixes process overload from file events #3175

Closed acedrew closed 2 months ago

acedrew commented 2 months ago

Description

Changes file watcher classes to use "on_closed" rather than "on_any_change" event type, reducing frequency of events. Fixes #3172

Type of change

Please delete options that are not relevant.

[*] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Deployed to platforms that were experiencing extreme resource utilization and verified utilization dropped significantly
[ ] Deployed to platforms with ongoing system profiling and verified the hotpaths in the platforms changed.

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code

craig8 commented 2 months ago

@acedrew This actually breaks the reloading of the web-users.json file.

acedrew commented 2 months ago

Why does it break that particular file and not any of the other ones?

On Mon, May 6, 2024, 1:18 PM Craig @.***> wrote:

@acedrew https://github.com/acedrew This actually breaks the reloading of the web-users.json file. However I found a fix, remove gevent_watchdog and just use watchdog. I am testing now to make sure it doesn't have any other concequences.

— Reply to this email directly, view it on GitHub https://github.com/VOLTTRON/volttron/pull/3175#issuecomment-2096534014, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHOUB3KCGRWLL2VN3ZUIZLZA63M5AVCNFSM6AAAAABGTN2WAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJWGUZTIMBRGQ . You are receiving this because you were mentioned.Message ID: @.***>

craig8 commented 2 months ago

I don't know...my guess is that we are reading auth.json in more than we do the web. So there are places that read it regardless of it being actually run through the event itself. Web users is only read/written in a couple of places.

craig8 commented 2 months ago

I suppose the other thing could be some gevent stuff happening that unless you use all_events then it's not getting propagated correctly? Not 100% sure

craig8 commented 1 month ago

@acedrew I think I found the issue. So the webusers uses a PersistentDict to write the data to a file. It writes data to a temp file and the copies the file into the proper space. This apparently is not what the filewatch mechanisms is expecting so it sends an infinite list of opened events! Sigh!