search

xavier-hernandez / goaccess-for-nginxproxymanager

GoAccess Docker Image for Nginx Proxy Manager and more...
https://hub.docker.com/r/xavierh/goaccess-for-nginxproxymanager
MIT License
548 stars 38 forks source link

Support custom config file #57

Open FoxxMD opened 2 years ago

FoxxMD commented 2 years ago

It's great that you've updated/built upon GregYankovoy's image but it does not look like the goaccess.conf file is user-overwritable which means anyone NOT using NPM is unable to use this.

I would suggest:

Move geoip db

COPY the db into /var so its unlikely to be overwritten by mounting a volume (and update the default config to reflect the location change)

Copy default config on container start instead of in build

Instead of COPY the NPM config into /goaccess-config during build, COPY it somewhere else as a backup and then during startup check for the file and if it does not exist then copy it to the required location.

This will make it so:

  1. users can mount a host directory to /goaccess-config that contains a custom config
  2. if no config exists the default one is created in the host's directory which allows them to edit it themselves

Since the config in the current image cannot be modified by the user you should be able to make these changes without any compatibility issues for current users.

xavier-hernandez commented 2 years ago

Hi @FoxxMD ,

Actually, I just closed a ticket I had opened for myself, from the beginning of the project, to allow the mounting of the config. The problem I was noticing while supporting this project is that it would be harder to assist or debug with users having a custom config because even now it's sometimes difficult.

If you are interested in a project closer to GregYankovoy's I can probably do that as separate project/repo.

Thanks.

FoxxMD commented 2 years ago

I could fork and do it myself...was just hoping to keep efforts consolidated, especially for the unraid community apps (not wanting to add another goaccess template)

xavier-hernandez commented 2 years ago

@FoxxMD let me take a look and see what I can do.

xavier-hernandez commented 2 years ago

did you want the geoip db mountable as well?

FoxxMD commented 2 years ago

It shouldn't matter, as long as you still copy the default database and use it in the default config.

Since the config would be user-configurable the user could mount an arbitrary folder with their own db and point to it using their config, completely bypassing the included one.

The problem I was noticing while supporting this project is that it would be harder to assist or debug with users having a custom config because even now it's sometimes difficult.

This is a fair point but you could include a disclaimer in the readme about not supporting custom configurations and instead direct users to the main goaccess repo.

FoxxMD commented 2 years ago

I'm happy to make a PR for this if you'd like.

xavier-hernandez commented 2 years ago

I'm actually doing a bunch of changes to the code to support another instance of goaccess running in the container so let me do it. The other instance will parse the redirection logs from npm.

xavier-hernandez commented 2 years ago

@FoxxMD a custom config is now supported in the develop branch and image.

docker pull xavierh/goaccess-for-nginxproxymanager:develop

Feel free to test. Please review the readme file for more information. You'll need to set an environment flag LOG_TYPE of custom and pass in a new volume that will contain the config file.

FoxxMD commented 2 years ago

Thanks for the hard work!

I am not able to get it read my config file...

GOAN v1.1.1

NGINX SETUP...

NGINX BASIC AUTH
-------------------------------
None

CUSTOM
-------------------------------
goaccess.conf file found
goaccess.conf not readable

I've verified it should be at the very least readable by anyone image

I've tried to use PUID and PGID correctly (setting to nobody/users) as well as trying with root with no change. Also have DEBUG=True but do not get any other output.

I've also tried using the default conf from your repo but had no success (same output).

How can I tell if this is a permission issue or a problem with my config file (line endings or something?)

xavier-hernandez commented 2 years ago

@FoxxMD it doesn't sound like a line-ending issue. It found the file. The bash script is checking if the file has read permissions.

if [[ -r ${goan_config} ]]; then echo -e "goaccess.conf readable" else echo -e "goaccess.conf not readable" exit fi

xavier-hernandez commented 2 years ago

@FoxxMD let me see if i can add additional debugging

xavier-hernandez commented 2 years ago

@FoxxMD I just updated the develop image and code. Please pull the latest develop image.

docker pull xavierh/goaccess-for-nginxproxymanager:develop

xavier-hernandez commented 2 years ago

You should see something like this.

CUSTOM
-------------------------------
goaccess.conf file found

**************BEGIN DEBUG***********************
/opt/custom/goaccess.conf file permissions
644 root root

/opt/custom/goaccess.conf content
######################################
# Time Format Options (required)
######################################
#.....
FoxxMD commented 2 years ago

@xavier-hernandez thanks for the quick and changes!

When using the correct user/group for my filesystem (nobody=99, users=100) PUID=99 PGID=100: 

GOAN v1.1.2
NGINX SETUP...
NGINX BASIC AUTH
-------------------------------
None
CUSTOM
-------------------------------
goaccess.conf file found
**************BEGIN DEBUG***********************
/opt/custom/goaccess.conf file permissions
777 users UNKNOWN
/opt/custom/goaccess.conf content
######################################
# Time Format Options (required)
######################################
#
...(omitted contents for brevity)
**************END DEBUG***********************
goaccess.conf not readable

When using PUID=0 PGID=0 (root/root) or I do not set them at all:

GOAN v1.1.2
NGINX SETUP...
NGINX BASIC AUTH
-------------------------------
None
CUSTOM
-------------------------------
goaccess.conf file found
**************BEGIN DEBUG***********************
/opt/custom/goaccess.conf file permissions
777 users UNKNOWN
/opt/custom/goaccess.conf content
######################################
# Time Format Options (required)
######################################
#
...(omitted contents for brevity)
**************END DEBUG***********************
goaccess.conf not readable

So the uid/gid for the user in the container seem unchanged

xavier-hernandez commented 2 years ago

Are the file contents still listed?

FoxxMD commented 2 years ago

Yes they are.

I may be missing it but I also don't see anywhere where you are using PUID/PGID in the dockerfile or startup scripts. Changing those is not something built into the alpine base image.

You could easily migrate to the linuxserver.io base image (dockerhub) which would take care of that for you, though. I use it for the base of one of my projects to handle puid/pgid.

xavier-hernandez commented 2 years ago

@FoxxMD - this "777 users UNKNOWN" is the permission for the file not for the PUID or PGUID. You are correct in that I'm setting the PUID and PGUID correctly but since I'm not writing anything to a mount this hasn't come up as an issue. Maybe the problem here is that bash script is running as a different user within docker. I'll look into it.

But for now if I remove the exit command from the container it should work for you since it is "readable".

FoxxMD commented 2 years ago

Getting much closer now with the new commit, thanks!

...(config omitted for brevity)
**************END DEBUG***********************
goaccess.conf not readable
DEBUG
-------------------------------
ON
RUN CUSTOM GOACCESS
Daemonized GoAccess: 24

Web server is outputting Custom instance processing: (might take some time...) and nothing else. Been running for about 5 minutes now. CPU is idle. Nothing else in the docker logs.

xavier-hernandez commented 2 years ago

@FoxxMD anything show up? How many logs file did you add to the config?

FoxxMD commented 2 years ago

I went back and used the default config from the repo with only a few changes for my log format. The web server is still outputting custom instance processing... but the logs now have:

goaccess.conf not readable

DEBUG
-------------------------------
OFF

RUN CUSTOM GOACCESS
<!DOCTYPE html><html lang='en'><head><meta charset='UTF-8'><meta name='referrer' content='no-referrer'><meta http-equiv='X-UA-Compatible' content='IE=edge'><meta name='google' content='notranslate'><meta name='viewport' content='width=device-width, initial-scale=1'><meta name='robots' content='noindex, nofollow'><link rel='icon' href='data:image/x-.......

It looks like goaccess report is getting output to buffer?

xavier-hernandez commented 2 years ago

@FoxxMD would it be possible for you to share your goaccess.conf file?