Log files with wildcards stopped being ingested after upgrading to 1.6.1

[jancurn]

After upgrading the logdna-agent from 1.5.6 to version 1.6.1 (running on Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-1048-aws x86_64)), LogDNA stopped receiving logs from our systems. Here's an analysis from the terminal:

ubuntu@ip-172-31-57-147:/var/log$ cat /etc/logdna.conf 
logdir = /home/apifier/*.log,/var/log/nginx/error.log
key = [REDACTED]

ubuntu@ip-172-31-57-147:/var/log$ sudo systemctl start logdna-agent

ubuntu@ip-172-31-57-147:/var/log$ cat logdna-agent.log
[190918 09:50:34] logdna-agent 1.6.1 started on ip-172-31-57-147 (172.31.57.147)
[190918 09:50:34] Streaming /var/log/nginx/error.log: 1 file(s)

ubuntu@ip-172-31-57-147:/var/log$ ls /home/apifier/*.log
/home/apifier/app@prod-api0.log  /home/apifier/app@prod-api3.log     /home/apifier/newrelic_agent.log
/home/apifier/app@prod-api1.log  /home/apifier/app@prod-daemons.log  /home/apifier/system.log
/home/apifier/app@prod-api2.log  /home/apifier/app@prod-ui.log       /home/apifier/testground@prod.log

Basically, it seems LogDNA agent is no longer able to handle wildcards in logdir option, such as /home/apifier/*.log. When we update /etc/logdna.conf as follows:

logdir = /home/apifier/app@prod-api3.log,/home/apifier/*.log,/var/log/nginx/error.log
key = [REDACTED]

then streaming of the file app@prod-api3.log resumes normally.

[sidprak]

We're encountering this issue as well. It seems like nothing in /var/log is being streamed either because of this.

[andkon]

Thanks for your feedback – and apologies for the trouble. It looks like we pushed the image prematurely; a quick fix will be to revert to 1.5.6 while we address this issue and properly release 1.6.1: sudo apt-get install logdna-agent=1.5.6

I'll keep this issue open as it's something we'll need to resolve for the 1.6.1 release anyhow.

[jancurn]

Please, can you add some integration test for this, so that it doesn't happen again?

[andkon]

I hear you. As we're still digging in on what happened and why the release was pushed to our CDN prematurely, I'm not sure yet if an integration test would solve that – but I'm going to make sure we dig in on what exactly went wrong and find the most complete way to ensure it doesn't happen again.

[jancurn]

So was this fixed?

[andkon]

@jancurn it was! Mostly it was an ephemeral problem that showed up on your end because we screwed up and pushed 1.6.1 early. I wanted to verify this was for sure fixed in 1.6.2 before closing, and can confirm it does indeed.