Open tokar86a opened 7 months ago
Hi, i haven't had heard about project before. It definitely looks promising.
Have you tried running npmGrafStats as the project is a fork from the original NPM?
Are you able to provide a censored log entry from NPMPlus and the log location? Then i can have a look at the default output to logs.
Yes i have try and it is dealing with the log in a different way. Instead of different logs related to domain/proxy host it is all centralized to one access log. But it is not working as it is now as I have try.
The logs rotation is something that the user can change. Maybe try to spin it up and look for you self then you would easy get the date that you need. Or maybe ask the developer?
Hope this helps :)
This is how the log locks like.
[12/Mar/2024:07:06:08 +0100] tokar86a.top ip address 0.192 "GET / HTTP/2.0" 200 36737 37435 - Uptime-Kuma/1.23.11
So what do you think?
I haven't had to test it and haven't forgotten you.
Nit sure if i will get to it this weekend but probably over the Easter holidays.
Nice and thanks for the updated. No rush on this as i was more looking for on what you think about it.
It is your project after all and it don`t seems like allot of user use the NPMplus as it was new to you.
whats NPMPlus like ? i am a little curious, has it filled alot of NPM bugs, and shortfalls ? i did see its got a nice dark mode ! and talks more about the API, does it have more Stream options ?
whats NPMPlus like ? i am a little curious, has it filled alot of NPM bugs, and shortfalls ? i did see its got a nice dark mode ! and talks more about the API, does it have more Stream options ?
Have you check out there Github page? https://github.com/ZoeyVid/NPMplus
This is how the log locks like.
[12/Mar/2024:07:06:08 +0100] tokar86a.top ip address 0.192 "GET / HTTP/2.0" 200 36737 37435 - Uptime-Kuma/1.23.11
@tokar86a I just tried it to set it up. Maybe you can help me. I set LOGROTATE=true
but i only see the NPMplus dashboard logs appear in the /opt/npm/ngin/access.log file. Am i missing something?
This is how the log locks like. [12/Mar/2024:07:06:08 +0100] tokar86a.top ip address 0.192 "GET / HTTP/2.0" 200 36737 37435 - Uptime-Kuma/1.23.11
@tokar86a I just tried it to set it up. Maybe you can help me. I set
LOGROTATE=true
but i only see the NPMplus dashboard logs appear in the /opt/npm/ngin/access.log file. Am i missing something?
What you mean whit dashboards logs?
I use the following environment:
[10/Apr/2024:16:53:22 +0200] grafana.sysadmin222.top 172.68.243.31 0.006 "GET /api/ruler/grafana/api/v1/rules?subtype=cortex HTTP/2.0" 200 1826 2159 https://grafana.sysadmin222.top/alerting/grafana/f3936ead-3fee-45ef-a8db-d9bada2dfacb/view?returnTo=%2Falerting%2Flist Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0 [10/Apr/2024:16:53:24 +0200] grafana.sysadmin222.top 172.68.243.31 0.004 "GET /api/search?type=dash-db&query=&limit=100 HTTP/2.0" 200 1235 1559 https://grafana.sysadmin222.top/alerting/list Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0 [10/Apr/2024:16:53:24 +0200] grafana.sysadmin222.top 172.68.243.31 0.005 "GET /api/user HTTP/2.0" 200 295 619 https://grafana.sysadmin222.top/alerting/list Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0 [10/Apr/2024:16:53:24 +0200] grafana.sysadmin222.top 172.68.243.31 0.007 "GET /api/prometheus/grafana/api/v1/rules?limit_alerts=16 HTTP/2.0" 200 788 1112 https://grafana.sysadmin222.top/alerting/list Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0 [10/Apr/2024:16:53:24 +0200] grafana.sysadmin222.top 172.68.243.31 0.005 "GET /api/ruler/grafana/api/v1/rules?subtype=cortex HTTP/2.0" 200 1177 1501 https://grafana.sysadmin222.top/alerting/list Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0 [10/Apr/2024:16:53:25 +0200] grafana.sysadmin222.top 172.68.243.31 0.003 "GET /api/prometheus/a111cd43-df3e-434c-a508-b9861bde17d4/api/v1/rules HTTP/2.0" 200 45 369 https://grafana.sysadmin222.top/alerting/list Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0 [10/Apr/2024:16:53:27 +0200] grafana.sysadmin222.top 162.158.134.229 64.681 "GET /api/live/ws HTTP/1.1" 101 108 562 - Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0
That should give you above information in the access log file "access.log"
Thank you very much. That helped :). Supporting npmPlus will require a bit of rewriting but won't be too difficult. As i am not able to create a fork of my own project i will develop npmPlusGrafStats inside a new branch.
I will keep you updated.
Thank you very much. That helped :). Supporting npmPlus will require a bit of rewriting but won't be too difficult. As i am not able to create a fork of my own project i will develop npmPlusGrafStats inside a new branch.
I will keep you updated.
Maybe ask the developer about what going to change when it comes to logging. https://github.com/ZoeyVid/NPMplus/discussions/731
Seems like she is going to rework it into php. So maybe best to wait a little so you don't need to redo the work again?
Thanks for the notice. As it will stay nginx as proxy for now, I don't see a problem.
Thanks for the notice. As it will stay nginx as proxy for now, I don't see a problem.
Thats good to hear.
I have yesterday pushed the plus version: https://hub.docker.com/r/smilebasti/npmgrafstats/tags.
Would be great if you could test it.
Here is the docker-compose.yml file. You can leave out npmplus as you already got a running instance.
version: '3.8'
services:
npmplus:
container_name: npmplus
image: zoeyvid/npmplus
restart: always
network_mode: host
volumes:
- "/opt/npm:/data"
environment:
- "TZ=Europe/Berlin" # set timezone, required
- "LOGROTATE=true" # required for npmgrafstats
# for more see https://github.com/ZoeyVid/NPMplus/blob/develop/compose.yaml
npmgraf:
image: smilebasti/npmgrafstats:plus-latest # change to ghcr.io/smilebasti/npmgrafstats if you prefer Github Container Registry
restart: unless-stopped
environment:
INTERNAL_LOGS: '<set TRUE or FALSE>' # see Github wiki for more information
MONITORING_LOGS: '<set TRUE or FALSE>' # see Github wiki for more information
INFLUX_HOST: '<replace>:8086' # use host IP
INFLUX_BUCKET: 'npmgrafstats'
INFLUX_ORG: 'npmgrafstats'
INFLUX_TOKEN: '<replace>' # insert after first run and manual token creation
volumes:
- /opt/npm/nginx/access.log:/access.log
- ./geolite:/geolite
- ./monitoringips.txt:/monitoringips.txt # optional only mount if preexists and a wanted feature
depends_on:
- geoipupdate
- npmplus
influxdb:
image: influxdb:2.7-alpine
restart: unless-stopped
environment:
DOCKER_INFLUXDB_INIT_MODE: 'setup'
DOCKER_INFLUXDB_INIT_USERNAME: '<replace>'
DOCKER_INFLUXDB_INIT_PASSWORD: '<replace>' # atleast 8 characters
DOCKER_INFLUXDB_INIT_ORG: 'npmgrafstats'
DOCKER_INFLUXDB_INIT_BUCKET: 'npmgrafstats'
volumes:
# Mount for influxdb data directory and configuration
- ./influxdbv2:/var/lib/influxdb2
ports:
- '8086:8086'
geoipupdate:
image: maxmindinc/geoipupdate
restart: unless-stopped
environment:
GEOIPUPDATE_ACCOUNT_ID: '<replace>'
GEOIPUPDATE_LICENSE_KEY: '<replace>'
GEOIPUPDATE_EDITION_IDS: 'GeoLite2-City GeoLite2-ASN' #GeoLite2-ASN is optional
GEOIPUPDATE_FREQUENCY: 24
volumes:
- ./geolite:/usr/share/GeoIP
portainer:
image: portainer/portainer-ce:latest
restart: always
ports:
- '8000:8000'
- '9000:9000'
- '9443:9443'
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./portainer_data:/data
grafana:
image: grafana/grafana-oss
restart: unless-stopped
ports:
- '3000:3000'
volumes:
- ./grafana-storage:/var/lib/grafana # after start: sudo chown 472:472 grafana-storage
I am leaving out most of the part in this and only use following
version: "3.8" services: npmgraf: image: smilebasti/npmgrafstats:plus-latest # set tag to 'latest-arm' for arm/v7 and arm64 image container_name: npmstats restart: unless-stopped environment: REDIRECTION_LOGS: "TRUE" # TRUE or FALSE or ONLY INTERNAL_LOGS: "FALSE" # see Github wiki for more information MONITORING_LOGS: "FALSE" # see Github wiki for more information INFLUX_HOST: # use host IP INFLUX_BUCKET: npmgrafstats INFLUX_ORG: myorg INFLUX_TOKEN: # insert after first run and manual token creation volumes:
networks: {}
But it seems to be working good.
I have notes something now. When the log rotation happen it seems like i need to shutdown the container and start it backup again for it to work. So it seems to have trouble when the new file is being created. Something you have notes?
Ok interesting. It should continue to work as tail -F
follows the filename and retries if not found.
I will have a look into it. The branch is now available: https://github.com/smilebasti/npmGrafStats/tree/npmPlus-main
Ok interesting. It should continue to work as
tail -F
follows the filename and retries if not found.I will have a look into it. The branch is now available: https://github.com/smilebasti/npmGrafStats/tree/npmPlus-main
yes it sure seems strange. But have you check that you end up whit the same problem?
Haven't but will do.
Try again today and ended up whit the same problem.
I noticed the same thing. Will take a few days to test different solutions as deleting the file makes it work :)
are the log rotate issues you are having, due to them being disabled by default
Note: access.log/stream.log, logrotate and goaccess are NOT enabled by default because of GDPR, you can enable them in the compose.yaml.
are the log rotate issues you are having, due to them being disabled by default
Note: access.log/stream.log, logrotate and goaccess are NOT enabled by default because of GDPR, you can enable them in the compose.yaml.
What you mean? Have you not check my compose file?
https://github.com/smilebasti/npmGrafStats/issues/41#issuecomment-2058504099
Any update about this?
Sadly not. I have tried a few options of tail with -F or just -f or similar, but none of them worked. Also tried setting the log rotation to 0.
I currently don't have the time to test other options or rewrite the whole thing. If you want to try it, just download the scripts and test variations or if you have experience you could rewrite the code to help.
Sadly not. I have tried a few options of tail with -F or just -f or similar, but none of them worked. Also tried setting the log rotation to 0.
I currently don't have the time to test other options or rewrite the whole thing. If you want to try it, just download the scripts and test variations or if you have experience you could rewrite the code to help.
Thanks for the update and sadly not i don`t have any experience when it comes to this. It sure is strange as it working whit vanila NPM so it should not be allot different then.
Hi
Would be nice if this could work for NPMplus. Something you have been thinking about @smilebasti ?
https://github.com/ZoeyVid/NPMplus