search

lordmilko / PrtgAPI

C#/PowerShell interface for PRTG Network Monitor
MIT License
301 stars 37 forks source link

Low Free Memory on PRTG Core Server #253

Closed schoenm1 closed 2 years ago

schoenm1 commented 2 years ago

What's going on?

Hi @lordmilko We created a fully automated Script (Powershell) with your PRTGAPI Module. We create devices from an excel input. For each device, multiple sensors will be added: ping, system uptime, memory, cpu.... In PRTG Version 21.3.71.1416+ (not sure if PRTG Version Issue) the PRTG Core Server Free Memory will decrease until Server is unreachable and has to be restarted. (Server has >30GB Memory). Our Script will connect to PRTG Server, then ... Add-Group / Add-Device / Add-Sensor ... Do I need to free up any memory again after this commands? It seems that the used memory during this Add-XYZ Task are reserved for ever until the server runs out of memory. I tested more than one time. Each time I use PRTGAPI, the memory will decrease (in 10minute script memory from 46% free to 1% free)

Due Dilligance

lordmilko commented 2 years ago

Hi @schoenm1,

Previously I would have said this is a problem with PRTG rather than PrtgAPI specifically, and there's nothing that can be done about it. My general recommendation when doing "memory intensive" operations like this is to spread it out a bit, introducing artificial delays between each item that is processed (e.g. you could do sleep 60 to sleep for 60 seconds after each device that is added). By spreading things out more PRTG may be able to better deal with its memory issues.

Recently, Paessler alerted me to a secret parameter PrtgAPI can include in some of its API calls which should reduce memory usage, however much testing/exploration needs to be done on this so this is not something that will be incorporated into PrtgAPI any time soon. As such, for now I would say "slowing down your script" may be the best course of action

schoenm1 commented 2 years ago

Hi @lordmilko I had the same issue with 'Memory Leak' again and again. We splitted our large (>10K Sensors) PRTG Instances in two. But same again... I tried to reproduce the issue with paessler... With command from your PowerShell Module with '-verbose' command I've got an answer from Paessler...

Output E-Mail from Paessler: Thank you for the detailed feedback. I was now able to locate the problem: For example, you use the "Set-ChannelProperty" command, which calls the PRTG URL with "/editsettings" in the background (recognizable with '-verbose'). However, this URL ("/editsettings") is not an endpoint designed to be used as an API call (recognizable by "/api/" in the URL) [...] The problem here is that this URL opens a session that is never closed in this case. With many such calls, a memory leak occurs. As a workaround, "nosession=1" could be added to the "/editsettings" URL (although this would require modifying the Powershell module accordingly) - in this case the session will then be closed (as with API commands designed for this). In the new upcoming API this problem will be solved by API keys in the future.

Is there any possibility with using PRTGAPI PowershellModule to implement the workaround mentioned by paessler (nosession=1)?

lordmilko commented 2 years ago

Hi @schoenm1,

Paessler privately advised me about the nosession parameter last year, which I've implemented in a private branch. As such this will be included in a future PrtgAPI release

schoenm1 commented 2 years ago

Hi @lordmilko

  1. Is there a timeline the 'nosession' will be implemented in public Repository?
  2. Is there a way for me to add the 'nosession' as additional parameter or any other workaround until 'nosession' is added in public Repository?
lordmilko commented 2 years ago

Hi @schoenm1,

I have moved this commit into the public master branch

Please follow the manual installation instructions to use PrtgAPI 0.9.18-preview.6 which includes this change

schoenm1 commented 2 years ago

Hi @lordmilko Thanks a lot! I will try the release PrtgAPI 0.9.18-preview.6.

schoenm1 commented 1 year ago

I used PrtgAPI 0.9.18-preview.6 for some implementation. No PRTG server had the memory leak again. Free Memory decrease, but only ~3-5%, but, but not that much that the server is crashing. Thanks @lordmilko