Closed calestyo closed 8 years ago
Very odd. This is usually what I expect if the logback.xml file is tampered with. Since a restart cured it, this seems not to have been the case.
Probably nothing I can do to help investigate it. Should it ever happen again, a heap dump of the domain would be helpful.
Cheers,
/gerd
Uhm.. I do have a non default logback (remembering now, that you say it),... but it's not really special, just changing some file paths... (attached it)... and why would the issue then only happen on one node (all have the same logback.xml) logback.xml.txt
Well I just wanted to tell you... I'm fine if we close it.
Non default is okay as long as one remembers to adjust it if the default one got structural changes. Since a restart resolved it, I do not suspect this to be the problem in this case.
Hi.
Not sure if the following is helpful for you... dCache 2.15.3, a node which has just pool services and one openly available (ro) nfs door, that is however not used.
What happened was, that one of the pools filled up the system disk over the course of a few days; dCache itself seemed to continued running fine (or at least we had gotten no complaints from our users).
/var/log/dcache was:
The pool in question is apparently dc67_2,...
log.2.xz is basically full with these:
that continues until at some point in log.1, the following starts:
but due to the size I didn't really go through it, whether anything else of interest or changed patters are contained. Obviously the hexdumps seem to be files stored i dCache.
.Iog is basically full hexdumps as above but I also found the other messages I've had posted further above... again I didn't read through al the 140G (got tired at 120G ;) )... so there may be loads of other stuff in it which I just didn't see.
Logging settings in dCache are at default, and my colleagues promised me not to have played around without my knowledge...
A restart of dCache on that node seemed to have cured the issue (so far).
I still have all the logs, if you should want them I could in principle make them available for download, but then please tell me yes or no, because I'd rather like to delete them.
Cheers, Chris.