Graylog server time - no way to change

[jackdanpr]

This is what I have on System>Overview; User 2020-05-07 18:47:00 -4:00 Your web browser 2020-05-07 18:47:00 -4:00 Graylog server 2020-05-07 22:48:00 +0:00

That is an issue with the logs. I can only see the last 8 hours of data on Grafana.

I have changed the timezone on the server without luck. garylog.env is GRAYLOG_TIMEZONE=America/Puerto_Rico

Any ideas?

Thanks!

[liviuancas]

You can try to add TZ=Europe/Bucharest to the graylog.env file

[DANgerous25]

You can try to add TZ=Europe/Bucharest to the graylog.env file

I tried this, and whilst this worked and my date command now outputs correctly, I still have the wrong timezone configuration in Graylog.

graylog@4fe38a842e36:~$ env | grep TZ TZ=Asia/Hong_Kong

graylog@4fe38a842e36:~$ env | grep TIME GRAYLOG_TIMEZONE=Asia/Hong_Kong

User admin: 2020-06-06 07:02:49 +00:00 Your web browser: 2020-06-06 15:02:49 +08:00 Graylog server: User admin: 2020-06-06 07:02:49 +00:00 Your web browser: 2020-06-06 15:02:49 +08:00 Graylog server:

graylog@4fe38a842e36:~$ date Sat Jun 6 15:03:25 HKT 2020

graylog@4fe38a842e36:~$ cat /etc/timezone Etc/UTC

I am trying to get all the timezones in Asia/Hong_Kong. Date shows correct, but my /etc/timezone is still wrong. Any help would be appreciated.

[liviuancas]

I have not figured the graylog user part... but you can create a new administrator user and there you can change the timezone option for that new user.

[DANgerous25]

I've at least partially solved this issue, it remains to be seen if this works. I've managed to set the local time, and local system variables to represent my timezone. I've also edited the config file within the container to point to my timezone for the admin user. The strange thing is that the System/Configuration page still shows the incorrect timezones, however I am now getting log messages that I wasn't getting before, and they look to have the correct timestamps. Also on the admin user page, it shows the correct timezone. Here's what I did:

1. Update the docker config to set local variables (I am no expert so this is probably a hack)

Edit the file ~/pfsense-analytics/Docker/graylog/Dockerfile and add in the lines below the chmod command below:

FROM graylog/graylog:3.1
# Probably a bad idea, but it works for now
USER root
RUN mkdir -pv /etc/graylog/server/
COPY ./getGeo.sh /etc/graylog/server/
RUN chmod +x /etc/graylog/server/getGeo.sh && /etc/graylog/server/getGeo.sh
RUN apt-get install tzdata -y
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
ENV TZ="Asia/Hong_Kong"
RUN echo "Asia/Hong_Kong" > /etc/timezone
RUN dpkg-reconfigure -f noninteractive tzdata
RUN date
USER graylog

2. Delete any trace of the container and rebuild (I'm sorry - my first time using a container, I don't know how to do a simple edit without doing this, I'm happy to correct if someone can advise!)

sudo docker stop $(sudo docker ps -aq)
sudo docker rm $(sudo docker ps -aq)
sudo docker rmi $(sudo docker images -q)

3. Manually edit graylog.conf from a shell within the container to update the timezone

Get the container ID CONT=`sudo docker ps -a | grep graylog | awk '{print $1}'`

Launch a shell using the container ID

sudo docker exec -it $CONT /bin/bash

Edit the config file

sed -i 's/#root_timezone = UTC/root_timezone = Asia\/Hong_Kong/g' /usr/share/graylog/data/config/graylog.conf

Exit the shell, stop/start Graylog

exit
sudo docker-compose stop graylog
sudo docker-compose start graylog

[liviuancas]

if that TZ var does not work another thing you could have done is mount a volume like this

  - "/etc/timezone:/etc/timezone:ro"
  - "/etc/localtime:/etc/localtime:ro"

[DANgerous25]

if that TZ var does not work another thing you could have done is mount a volume like this

  - "/etc/timezone:/etc/timezone:ro"
  - "/etc/localtime:/etc/localtime:ro"

Thank you, but please could you elaborate exactly what this will do and how to do it?

Note that my "solution" above unfortunately doesn't work. The timestamps coming in are still in UTC.

From Graylog System -> Overview Screen (with default "Admin" user)

User admin:
2020-06-06 12:21:56 +00:00
Your web browser:
2020-06-06 20:21:56 +08:00
Graylog server:
2020-06-06 12:21:56 +00:00

From Graylog System -> Overview Screen (with new admin user created from the GUI and timezone set to Hong Kong)

User dan:
2020-06-06 20:31:49 +08:00
Your web browser:
2020-06-06 20:31:49 +08:00
Graylog server:
2020-06-06 12:31:49 +00:00

Host system:

dan@unraid-ubuntu-server:~/pfsense-analytics/Docker$ date
Sat Jun  6 20:22:15 HKT 2020

Container:

graylog@a9aa22bf68ba:~$ env | grep Asia
TZ=Asia/Hong_Kong
GRAYLOG_TIMEZONE=Asia/Hong_Kong

graylog@a9aa22bf68ba:~$ cat /etc/timezone
Asia/Hong_Kong

graylog@a9aa22bf68ba:~$ date
Sat Jun  6 20:26:23 HKT 2020

graylog@a9aa22bf68ba:~$ grep Asia /usr/share/graylog/data/config/graylog.conf
root_timezone = Asia/Hong_Kong

What am I missing?

[liviuancas]

As far as I know graylog will always use UTC check if your incoming messages are displayed at the correct timestamp.

as for the volumes in the docker-compose file you can add like this

volumes:
  - "/etc/localtime:/etc/localtime:ro"
  - "/etc/timezone:/etc/timezone:ro"

basically this is adding timezone and localtime of the vm where docker is running inside of the containers.

[DANgerous25]

Thank you @liviuancas, I appreciate your responses. I've finally solved this problem using a modified version of the fork from https://github.com/SalehAlolayan . The short version of how to solve it us to replace Asia/Hong_Kong with Etc/GMT-8 in all the environment variables before building the container.

There's one gotya here that people should be aware of: Hong Kong is actually UTC plus 8 hours (equivalent of GMT plus 8 hours). So I've spent the last hour or so putting in Etc/GMT+8 and pulling my hair out as the time is still coming out wrong and no obvious reason why. It turns out that there is nuance in how the timezones are specified for the "Etc" region, it needs to be written GMT-8 (minus) when it in fact means plus 8 hours. (see https://en.wikipedia.org/wiki/Tz_database#Area).

So in summary: Don't use the normal notation like Asia/Hong_Kong, instead use Etc/GMT-8 but check the wikipedia page to find the exact notation for what you're looking for...

Now onto the next problem!

[sivom]

simple solution

sudo docker-compose down update your graylog.env with, for example America/New_York TZ=Etc/GMT+4

sudo docker-compose build sudo docker-compose up -d

Still this is not a robust solution, as the DST changes you have to update the TZ again. I think better solution is to update into the graylog.conf with proper time zone, for ex: root_timezone=US/Eastern

[lephisto]

This was an ongoing annoyance with Graylog and Elasticsearch, which seems to be fixed now in Graylog 4.

[AnonymousWP]

I still have not figured out how to fix this, neither after reading this thread and many other pages. Could someone post an example Docker Compose file? I tried it with multiple variables, with and without volume etc.