search

timja / jenkins-gh-issues-poc-06-18

0 stars 0 forks source link

[JENKINS-56223] Jenkins discard-old-build caused open file handles #4282

Open timja opened 5 years ago

timja commented 5 years ago

 
After Jenkins upgraded from 2.46.2 to 2.138.4 on Jan 5. 2019,

I started to see many (deleted) open file handles.

Those (deleted) open file handles seem caused by "discard old build" rotate.

The "builds/.${old-build-number}" folder doesn't exist anymore,

but when using "lsof" command to check, it showed up:

sudo lsof -a -p $(pgrep -P $(sudo cat /var/run/jenkins/jenkins.pid)) | grep '(deleted)'

eg

java 34189 jenkins 5199r REG 252,0 1818111 17044616 /var/lib/jenkins/jobs/Projects/jobs/FortiClientEMS/jobs/br_6_Branch_Build/builds/.1394/log (deleted)

java 34189 jenkins 5200r REG 252,0 1813394 17052273 /var/lib/jenkins/jobs/Projects/jobs/FortiClientEMS/jobs/br_6_Branch_Build/builds/.1388/log (deleted)

java 34189 jenkins 5201r REG 252,0 1813394 17052273 /var/lib/jenkins/jobs/Projects/jobs/FortiClientEMS/jobs/br_6_Branch_Build/builds/.1388/log (deleted)

java 34189 jenkins 5212r REG 252,0 1815212 17053470 /var/lib/jenkins/jobs/Projects/jobs/FortiClientEMS/jobs/br_6_Branch_Build/builds/.1398/log (deleted)

Originally reported by totoroliu, imported from: Jenkins discard-old-build caused open file handles
  • assignee: nkns165
  • status: Open
  • priority: Major
  • resolution: Unresolved
  • imported: 2022/01/10
timja commented 5 years ago

samsun385:

Is there any update on this issue? Our system is having the same problem at the moment.

timja commented 3 years ago

xraynaud:

I finally found the root cause of my issue: A groovy script opens the output of some runs using "run.getLogReader()".
And the created reader is never closed. Therefore, file handle remains open.

Hope this can help for other people having the same issue...