gluster / project-infrastructure

Issues related to GlusterFs infrastructure components.
0 stars 0 forks source link

glusterfs-logs.tgz generated during regression run is empty #181

Open xhernandez opened 1 year ago

xhernandez commented 1 year ago

The build artifact glusterfs-logs.tgz that is generated during the execution of jenkins' job gh_centos7-regression only contains /var/log/messages. Inside /var/log/glusterfs there's only a single file named '.notar'.

Some examples:

xhernandez commented 1 year ago

Any updates on this one ? newer runs still don't have anything inside var/log/glusterfs and I can't analyze regression failures.

mscherer commented 1 year ago

Seems to still be a issue. That's curious, because nothing changed recently.

mscherer commented 1 year ago

Ok so checking the last build, it seems that something remove the log, then create the tarball.

See at the end of https://build.gluster.org/job/gh_centos7-regression/3200/console

16:33:50 + rm -rf /var/lib/glusterd/groups /var/lib/glusterd/hooks /var/log/glusterfs/.cmd_log_history '/etc/glusterd/*' '/var/log/glusterfs/*' /build/export

then

16:33:52 sudo -E tar -czf "${filename}" /var/log/glusterfs /var/log/messages*;
mscherer commented 1 year ago

That's here since 2014:

https://github.com/gluster/glusterfs-patch-acceptance-tests/commit/6a9f64fe0ee62b44b6a9243e4ad0102a19d733cc

So I am not sure why it was noticed only today. But the fix is to change that to not clean the log.

xhernandez commented 1 year ago

That's a bit weird because I'm pretty sure I've downloaded many archives during these years and the contents were there.

Just a recent example: https://build.gluster.org/job/gh_centos7-regression/3195/

This one contains data, though not everything it should be there.

In any case, the code should be changed.

mscherer commented 1 year ago

Ok so sometime, it work, and sometime it doesn't. Exactly the kind of problem I love to deal with ( sarcasm ).

I remember we had 2 different types of failure on jenkins, and I guess depending on the type of issue, it could go a different code path.

xhernandez commented 1 year ago

I'm not sure if it could help but other jobs, like centos8-regression, seem to always generate the tgz correctly (though the test itself fails constantly).