-f, --follow mode does not show any output

[jcudilla]

Hi, I installed bonfire only yesterday but I can't seem to get any output when using the -f nor --follow option. Wondering if I'm just doing it wrongly or perhaps I'm using a wrong syntax. Query works fine w/out the "follow" option but this flag is what I'm looking forward to use. For example:

• bonfire -@ "2021-07-11T23:30:00.831Z" -f "facility:* AND source:server1"
• bonfire -@ "2021-07-11T23:30:00.831Z" -f

No error whatsoever but no output either and I have to control+c to get back the prompt. Let me know if you have any tips, thanks..

[valentin-krasontovitsch]

hi, and thank you for using bonfire : )

i'm afraid i'm having a hard time reproducing your problem - which might be because my pip foo is not very good : /

i'm assuming you're running version 0.0.8 if you installed the day before. would you mind confirming by running

python3 -m pip show bonfire

besides that, i'm not sure, maybe your timestamp is the problem? would you mind trying something like

bonfire -@ '1 hour ago' -f

or perhaps

bonfire -@ '1 day ago' -f

[jcudilla]

Thanks for your suggestion, this is how it looks:

[root@client1 bonfire]# python3 -m pip show bonfire Name: bonfire Version: 0.0.8 Summary: Add a small description here! Home-page: http://... Author: Malte Harder Author-email: malte.harder@blue-yonder.com License: new BSD Location: /usr/local/lib/python3.6/site-packages Requires: arrow, keyring, termcolor, parsedatetime, python-dateutil, click, requests, six Required-by: [root@client1 bonfire]# bonfire -@ '1 hour ago' -f Enter password for user@IP.IP.IP.IP:9000/api:

^C Interrupted follow mode. Exiting... [root@client1 bonfire]#

I had to ctrl+c only to get back the prompt.

[valentin-krasontovitsch]

hmm that's disconcerting. let me think a bit about it, try out some stuff, and get back to you

could you give me an example query / invocation that produces logs, without follow mode?

[jcudilla]

sure, this is how it kind of looks w/out the follow flag:

[root@client1 bonfire]# bonfire -@ '1 day ago' Enter password for user@IP.IP.IP.IP:9000/api: NOTICE [2021-07-12 19:41:25.00] client2 python: detected unhandled Python exception # source:client2; facility:user-level; line:; module: INFO [2021-07-12 19:41:26.00] client3 nvexporter: source:client3; facility:system daemon; line:; module: NOTICE [2021-07-12 19:41:27.00] client4 source:client4; facility:user-level; line:; module: WARNING [2021-07-12 19:41:27.00] client4 source:client4; facility:user-level; line:; module: ERROR [2021-07-12 19:41:27.00] client4 launchd[1]: Service could not initialize INFO [2021-07-12 19:41:27.00] client5 Service could not initialize: Unable to set current working directory; facility:user-level; line:; module: DEBUG [2021-07-12 19:41:28.00] client2 facility:security/authorization; line:; module: INFO [2021-07-12 19:41:32.00] client2 nvexporter: 2021/07/12 19:41:32; facility:system daemon; line:; module: INFO [2021-07-12 19:41:34.00] client3 agent: kbuilds running... # source:client3; facility:system daemon; line:; module: INFO [2021-07-12 19:41:34.00] client3 facility:system daemon; line:; module: [root@client1 bonfire]#

[valentin-krasontovitsch]

ok so first off - sorry, my first suggestion of running with the -@ option was nonesense, in follow mode you can't set a before time... follow mode will always start following from one second ago...

secondly, probably you have truncated the output of bonfire, i.e. shortened the logs that you pasted here, and thank you for that, but perhaps you don't have that many logs appearing in graylog? i dunno... are you sure that there are logs being sent to graylog while you're following? probably you are, silly question, i just have to ask.

finally, i'm working on some new features in a new release, and i've created some release candidates.

perhaps you want to give those a spin - among other things, follow mode starts from 10 minutes ago, making it easier to see logs appear.

you can give it a spin by running

python3 -m pip install [--user] bonfire-1.0.0rc2

the --user option is of course optional, if you just want to install it for your own user, instead of system wide, while trying it out... otherwise run without --user and probably with sudo : )

let me know how it goes!

[jcudilla]

Thanks but I got this error using that specific version Could not find a version that satisfies the requirement bonfire-1.0.0rc2 (from versions: ) No matching distribution found for bonfire-1.0.0rc2

On the other hand, you were asking about the log(on web UI, we can see logs getting ingested almost every few seconds), Using bonfire, I just realize I cannot see the last few hours trying bonfire -@ '1 hour ago' up to bonfire -@ '6 hour ago' but occasionally seeing some output on bonfire -@ '7 hour ago' then finally seeing the normal last 10 lines when using bonfire -@ '8 hour ago'

which makes me wonder if it has something to do with the timezone and if I should switch back the server to the default UTC?

referenced using https://www.joda.org/joda-time/timezones.html, this is the server time

[root@graylogserver ~]# grep root_timezone /etc/graylog/server/server.conf
root_timezone = America/Vancouver
[root@graylogserver ~]#

and on the UI system/overview, this is how it looks:

User $user:     2021-07-18 10:45:08 -07:00
Your web browser:   2021-07-18 10:45:08 -07:00
Graylog server:     2021-07-18 10:45:08 -07:00

What do you think, any suggestions?

[valentin-krasontovitsch]

ah, right, right, that might be the source of the problems - you see, it seems that bonfire expects graylog's timezone to be UTC : /

is it possible for you to change the timezone of your graylog server?

if not, i should investigate this limitation and see if it can be fixed!

re: no matching distribution found - that's very strange, i tried installing it, and it worked, and you can even find it on the pypi - https://pypi.org/project/bonfire/#history

sadly i know too little about pypi and pip to offer any suggestions to troubleshoot / debug there...

but of course the newer version won't help you at all with the timezone problem (yet)..

[jcudilla]

Thanks for your suggestion, I tried changing root_timezone = UTC (or comment out) and it took care of the user time in the graylog UI but still I'm not able to use -f option and only able to see using bonfire -@ '8 hour ago'

User $user:     2021-07-22 04:40:23 +00:00
Your web browser:   2021-07-21 21:40:23 -07:00
Graylog server:     2021-07-21 21:40:23 -07:00

We then, tried changing the server/OS time to UTC via timedatectl tool which updates /etc/localtime.

User $user:     2021-07-22 04:49:49 +00:00
Your web browser:   2021-07-21 21:49:49 -07:00
Graylog server:     2021-07-22 04:49:49 +00:00

This finally made me see bonfire -@ '1 hour ago' but the server seems to have stopped ingesting client logs hence I'm not able to see incoming logs both on the UI and bonfire -f option. After further service restarts and checking, we ended up reverting to America/Vancouver timezone.

regarding 1.0.0rc2, I also don't know much about pydev nor pip but thanks for your confirmation. I'll ask around, it could be just me :sweat_smile:

[root@client1 ~]# curl -L -s "https://pypi.org/pypi/bonfire/json"| jq  -r '.releases | keys | .[]' | sort -V
0.0.5
0.0.6
0.0.7
0.0.8
0.0.8b1
1.0.0rc1
1.0.0rc2
[root@client1 ~]# python3 -m pip install bonfire-1.0.0rc2
ERROR: Could not find a version that satisfies the requirement bonfire-1.0.0rc2 (from versions: none)
ERROR: No matching distribution found for bonfire-1.0.0rc2
[root@client1 ~]#

[valentin-krasontovitsch]

OK, I'm definitely confused and not certain about how time zone settings in the server work, and how they effect consuming log messages, and so on, so i should read / figure out more about that to be able to be of assistance there.

i will however, as it seems more tangible (although i might still have to understand the server side things in order to test it), try to fix the timezone requirement in the client - and perhaps abuse you for testing ; )

give me some time to figure it out, please. i have a hunch that it should be easy, but you never know....

also, about the pip invocation - i think the correct syntax for installing a specific version of a package with pip is to use == rather than a simple dash between package name and verison, as in

python3 -m pip install bonfire==1.0.0rc2

see if that helps!

gonna let you know here when i have a timezone aware release candidate ready for testing.

[jcudilla]

python3 -m pip install bonfire==1.0.0rc2 worked but with the same behavior so I may need to wait for the timezone aware release, thank you..

[valentin-krasontovitsch]

so sorry i completely forgot about this as i went into parental leave... gonna try to take a look next week!