amidaware / tacticalrmm

A remote monitoring & management tool, built with Django, Vue and Go.
https://docs.tacticalrmm.com
Other
3.08k stars 436 forks source link

An abnormally large number of event logs prevents TacticalAgent from querying or viewing the event logs #1404

Open NiceGuyIT opened 1 year ago

NiceGuyIT commented 1 year ago

Server Info (please complete the following information):

Installation Method:

Agent Info (please complete the following information):

Describe the bug Due to various reasons, the event logs may contain thousands or tens of thousands of records. With a large number of records, TacticalRMM returns an error: 400 bad request

To Reproduce Steps to reproduce the behavior:

  1. Find a system that has 10's of thousands of events. One such way is explained on a Discord thread.
  2. Remote Background > Event Log
  3. Add a search term to query the logs.
  4. If necessary, increase the number of days.
  5. Tactical will return "400 bad request" and "Unable to contact agent" error messages.

Expected behavior It would be nice if Tactical handled the situation gracefully. This could be done using pagination for the events, or perform the search on the agent instead of the server (or browser?). A simple solution is to return the first X records with a message that the rest were truncated due to volume. If the error is due to timeout, maybe provide a message that the agent timed out after X seconds.

Screenshots Here's the error message when showing the last 1 days. It's unclear if it's due to the number of events or a timeout when gathering the events. image

Additional context A better error message would help with the troubleshooting effort.

As explained in the Discord thread above, querying Win32_Product will cause a bunch of "Windows Installer reconfigured the product" messages in the event logs. This is explained in Microsoft's KB articles. The suggestion to use Win32reg_AddRemovePrograms is not a perfect replacement as that causes an error. There currently are 3 scripts that use Win32_Product.

NiceGuyIT commented 1 year ago

Possibly related NATS errors in the agent.log.

time="2023-01-16T09:21:12-08:00" level=error msg="NATS error: nats: Permissions Violation for Publish to \"_INBOX.**********************.**********************06e0\""
time="2023-01-16T09:21:12-08:00" level=error msg="<nil>\n"
time="2023-01-16T09:21:16-08:00" level=error msg="NATS error: nats: Permissions Violation for Publish to \"_INBOX.**********************.**********************ed6c\""
time="2023-01-16T09:21:16-08:00" level=error msg="<nil>\n"
time="2023-01-16T09:28:14-08:00" level=error msg="NATS error: nats: Permissions Violation for Publish to \"_INBOX.**********************.**********************8c7d\""
time="2023-01-16T09:28:14-08:00" level=error msg="<nil>\n"
time="2023-01-16T09:38:40-08:00" level=error msg="NATS error: nats: Permissions Violation for Publish to \"_INBOX.**********************.**********************90be\""
time="2023-01-16T09:38:40-08:00" level=error msg="<nil>\n"
time="2023-01-16T09:39:00-08:00" level=error msg="NATS error: nats: Permissions Violation for Publish to \"_INBOX.**********************.**********************182a\""
time="2023-01-16T09:39:00-08:00" level=error msg="<nil>\n"