search

bb-Ricardo / check_redfish

A monitoring/inventory plugin to check components and health status of systems which support Redfish. It will also create a inventory of all components of a system.
MIT License
115 stars 34 forks source link

Added option --sessionlock which prevents multiple sessions and locks the session file when connecting #132

Closed itsul closed 3 months ago

itsul commented 3 months ago

Overview This update addresses an issue encountered when running the check_redfish.py script in parallel with different modes (e.g., storage, processor, memory). During initialization, multiple instances of the script may attempt to establish connections simultaneously, leading to an excessive number of open sessions. This happens because the session file is only written after a successful connection (which takes a while) and other instances can overwrite it. This result in reaching the session limit of an Integrated Lights-Out (iLO) interface or similar management interfaces.

Problem When the session limit is reached, no more logins into the management interface are possible. Also every subsequent check attempt creates additional session requests, potentially leading to more failures and unexpected behavior.

Solution To mitigate this issue, i added a lock file mechanism. This lock file ensures that multiple instances of the script do not open new sessions concurrently if a session is already in the process of being established. By coordinating access to session creation, this approach prevents exceeding the session limit and maintains system stability. I made this parameter optional so if not needed or wanted no lock file will be used.

bb-Ricardo commented 3 months ago

Good point. Thank you for this PR. Will have a closer look next week.

bb-Ricardo commented 3 months ago

Hi @itsul,

I merged it into next-release and updated some bits of the code. Would you be able to test the next-release branch again?

Thank you

itsul commented 2 months ago

Hi @bb-Ricardo ,

thanks. I did today some testing. I can't say for certain that the lock file works. Previous I did verify this successful. I have to investigate this deeper. May take a few weeks because I have some open tasks.

Regards Marcel

itsul commented 1 month ago

Hi @bb-Ricardo ,

thank you for your patience, unfortunately I didn't get to it earlier. I deeply tested it today and found the issue. I created a pull request #138 to solve it. With this change all my tests are successful.

Regards Marcel