search

smilebasti / npmGrafStats

This project exports NginxProxyManager Logs to InfluxDB for displaying them in a Grafana Dashboard.
GNU General Public License v3.0
149 stars 10 forks source link

Stops Working after a few Days #2

Closed smilebasti closed 1 year ago

smilebasti commented 1 year ago

Odd behavior that the image stops working after few days.

smilebasti commented 1 year ago

A workaround has come with v2.2.0. Restarting the container now doesn't duplicate the logs.

You can create a cron job to restart npmgrafstats to avoid that the collection stops.

smilebasti commented 1 year ago

If somebody has any idea with this issue or can help please let me know :)

G3VV commented 1 year ago

Still no update on this? I'm having this issue now every 5 minutes. Hit's slowly drop from their peak to "No Data", this was fixed with a cron. What the cron does not fix is the spikes of hit's per minute due to each restart.. image

smilebasti commented 1 year ago

@G3VV Sorry still not. Every 5 minutes seems to be too quick for my liking. In my use it happens about every 7 days with about 50 domains. The vm has for testing 4 GB Ram. Can you share something about your system? Maybe i can narrow it down a bit better? Ram usage and number of domains?

I will have a look at the spikes.

G3VV commented 1 year ago

@G3VV Sorry still not. Every 5 minutes seems to be too quick for my liking. In my use it happens about every 7 days with about 50 domains. The vm has for testing 4 GB Ram. Can you share something about your system? Maybe i can narrow it down a bit better? Ram usage and number of domains?

I will have a look at the spikes.

My proxy server is 8GB with 4 cores, i have 210 proxy hosts. About 5 minutes after restarting the npmgraf docker the "Hit's in the last 5 minutes" will drop to "No Data" along-side all other information ceasing to update? I've looked at logs on all services and nothing shows any errors? image image

bam91 commented 1 year ago

You can create a cron job to restart npmgrafstats to avoid that the collection stops.

Is there a nice way to do so in Docker?

I have checked the code and thought about it. My first Idea: The problem could be in https://github.com/smilebasti/npmGrafStats/blob/main/Getipinfo.py#L122 . The Syncronus write would maye stop the whole process if Influxdb is not responding. Solution: Set some kind of timeout to this operation or make it async. Async would be easy to do (https://github.com/smilebasti/npmGrafStats/blob/main/Getipinfo.py#L95). Maybe you could try this and add it to the image.

smilebasti commented 1 year ago

@bam91 Thanks for the ideas. I will test them.

I am also currently testing if it's a problem with not closing the connection after the data is written to InfluxDB. Will update when something works.

smilebasti commented 1 year ago

Closing the connection to InfluxDB after the data was sent seemed to help.

Now I had it stop working when the logs rotated. I will try tail -F instead of tail -f. That will be easier to test than the connection limit.

If it works I can push a preview/test image with the fix in a couple of days.

smilebasti commented 1 year ago

I released v2.3.1. It should fix the need of restarting npmGrafStats.

Just pull the latest image and recreate npmGrafStats. Please let me know if we can close this issue :)

@G3VV i haven't done anything about the spikes. Please also let me know if they persist.

smilebasti commented 1 year ago

With no feedback and my experience that npmgrafstats now runs without interruptions i will close this issue.

Don't hesitate to comment/feedback on this issue or just open a new one.