Do not consider there are only Caddy files in the log folder

[rgaudin]

I'm seeing this in the compose log. Is this normal? If it is, maybe we should skip the error message

metrics        | [2024-01-03 15:01:23,553: DEBUG] in-event <InotifyEvent: src_path=b'/var/log/host/metrics/lastlog', wd=1, mask=IN_OPEN, cookie=0, name='lastlog'>
metrics        | [2024-01-03 15:01:23,553: DEBUG] in-event <InotifyEvent: src_path=b'/var/log/host/metrics/wtmp', wd=1, mask=IN_OPEN, cookie=0, name='wtmp'>
metrics        | [2024-01-03 15:01:23,555: DEBUG] in-event <InotifyEvent: src_path=b'/var/log/host/metrics/wtmp', wd=1, mask=IN_MODIFY, cookie=0, name='wtmp'>
metrics        | [2024-01-03 15:01:23,557: DEBUG] in-event <InotifyEvent: src_path=b'/var/log/host/metrics/wtmp', wd=1, mask=IN_CLOSE_WRITE, cookie=0, name='wtmp'>
metrics        | [2024-01-03 15:01:23,560: DEBUG] in-event <InotifyEvent: src_path=b'/var/log/host/metrics/lastlog', wd=1, mask=IN_OPEN, cookie=0, name='lastlog'>
metrics        | [2024-01-03 15:01:23,561: DEBUG] in-event <InotifyEvent: src_path=b'/var/log/host/metrics/lastlog', wd=1, mask=IN_MODIFY, cookie=0, name='lastlog'>
metrics        | [2024-01-03 15:01:23,561: DEBUG] in-event <InotifyEvent: src_path=b'/var/log/host/metrics/lastlog', wd=1, mask=IN_CLOSE_WRITE, cookie=0, name='lastlog'>
metrics        | [2024-01-03 15:01:23,563: DEBUG] in-event <InotifyEvent: src_path=b'/var/log/host/metrics/wtmp', wd=1, mask=IN_OPEN, cookie=0, name='wtmp'>
metrics        | [2024-01-03 15:01:23,565: DEBUG] Error occured while processing event modified on /var/log/host/metrics/wtmp
metrics        | Traceback (most recent call last):
metrics        |   File "/usr/local/lib/python3.11/site-packages/offspot_metrics_backend/business/log_watcher.py", line 85, in on_any_event
metrics        |     self.process_event(event)
metrics        |   File "/usr/local/lib/python3.11/site-packages/offspot_metrics_backend/business/log_watcher.py", line 110, in process_event
metrics        |     self.process_new_lines(Path(event.src_path))
metrics        |   File "/usr/local/lib/python3.11/site-packages/offspot_metrics_backend/business/log_watcher.py", line 60, in process_new_lines
metrics        |     new_data = file.read()
metrics        |                ^^^^^^^^^^^
metrics        |   File "<frozen codecs>", line 322, in decode
metrics        | UnicodeDecodeError: 'utf-8' codec can't decode byte 0xeb in position 344: invalid continuation byte
metrics        | [2024-01-03 15:01:23,570: DEBUG] Error occured while processing event modified on /var/log/host/metrics/lastlog
metrics        | Traceback (most recent call last):
metrics        |   File "/usr/local/lib/python3.11/site-packages/offspot_metrics_backend/business/log_watcher.py", line 85, in on_any_event
metrics        |     self.process_event(event)
metrics        |   File "/usr/local/lib/python3.11/site-packages/offspot_metrics_backend/business/log_watcher.py", line 110, in process_event
metrics        |     self.process_new_lines(Path(event.src_path))
metrics        |   File "/usr/local/lib/python3.11/site-packages/offspot_metrics_backend/business/log_watcher.py", line 60, in process_new_lines
metrics        |     new_data = file.read()
metrics        |                ^^^^^^^^^^^
metrics        |   File "<frozen codecs>", line 322, in decode
metrics        | UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 296000: invalid continuation byte
metrics        | [2024-01-03 15:01:23,570: DEBUG] in-event <InotifyEvent: src_path=b'/var/log/host/metrics/lastlog', wd=1, mask=IN_OPEN, cookie=0, name='lastlog'>

[benoit74]

This is indeed not expected. I wonder why you get this, if this is some kind of random error because the file is not yet fully flushed?

But what are these /var/log/host/metrics/wtmp and /var/log/host/metrics/lastlog files, this does not look like regular Caddy files?

[rgaudin]

Oh non those are not caddy files. I expected metrics to filter the files on the passed folder for only the caddy ones (at least JSON maybe?)

I can not mount a caddy-logs dedicated folder because I want to stay on tmpfs which is cleared on shutdown. We can discuss an alternate strategy if you want.

[benoit74]

Let's see, I just took the fast path, supposing there will only be caddy files in this directory, probably not that complex to change, I will tell you otherwise.

[benoit74]

These are the files that we expect to have:

-rw-r--r--    1 root     root     1023.3K Jan  9 13:43 caddy_access_logs-2024-01-09T13-43-07.371.json
-rw-r--r--    1 root     root       17.8K Jan  9 13:43 caddy_access_logs.json