search

jenkinsci / lib-file-leak-detector

Java agent that detects file handle leak
http://file-leak-detector.kohsuke.org/
MIT License
245 stars 112 forks source link

readme

= File Leak Detector

== What is this?

This little Java agent is a tool that keeps track of where/when/who opened files in your JVM. You can have the agent trace these operations to find out about the access pattern or file descriptor leaks, and dump the list of currently open files and where/when/who opened them.

In addition, upon a "too many open files" exception, this agent will dump the list, allowing you to find out where a large number of file descriptors are in use.

For a long running application, you can have it run a mini HTTP server to access the information on demand. This tool can be also loaded as a regular dependency JAR, and the file descriptor table can be https://javadoc.jenkins.io/component/file-leak-detector/org/kohsuke/file_leak_detector/Listener.html[programmatically accessed].

== Download

Download File Leak Detector from https://repo.jenkins-ci.org/releases/org/kohsuke/file-leak-detector/[the Jenkins repository].

== Usage

Without any options, this tool silently records file open/close operations and upon a "too many open files" exception, the file descriptor table will be dumped to standard error.

[source,sh]

$ java -javaagent:path/to/file-leak-detector-jar-with-dependencies.jar ...your usual Java arguments follow...

There are several options you can pass to the agent. For example, to dump the open file descriptors when the total number reaches 200, you can do the following:

[source,sh]

$ java -javaagent:path/to/file-leak-detector-jar-with-dependencies.jar=threshold=200 ...your usual Java arguments follow...

Continuous logging of every opened/closed file:

[source,sh]

$ java -javaagent:path/to/file-leak-detector-jar-with-dependencies.jar=trace=/path/to/logfile.txt ...your usual Java arguments follow...

Or to have it run a mini HTTP server so that you can access the information from your browser, do the following and open http://localhost:19999/:

[source,sh]

$ java -javaagent:path/to/file-leak-detector-jar-with-dependencies.jar=http=19999 ...your usual Java arguments follow...

Use the help option to see the help screen for the complete list of options:

[source,sh]

$ java -javaagent:path/to/file-leak-detector-jar-with-dependencies.jar=help

== Attaching after JVM startup

When run as a regular JAR file, this tool can be used to attach the detector into other JVMs on the same system. You specify the JVM by its PID. The following example attaches File Leak Detector to PID 1500. Options can be specified in the second argument in the same format you do to the agent.

[source,sh]

$ java -jar path/to/file-leak-detector-jar-with-dependencies.jar 1500 threshold=200,strong

== Supported options

File leak detector arguments (to specify multiple values, separate them by ',':
  help           - Show the help screen.
  noexit         - Don't exit after showing the help screen.
  trace          - Log every open/close operation to stderr.
  trace=FILE     - Log every open/close operation to the given file.
  error=FILE     - If 'too many open files' error is detected, send the dump here.
                   By default it goes to stderr.
  threshold=N    - Instead of waiting until 'too many open files', dump once
                   we have N descriptors open.
  http=PORT      - Run a mini HTTP server that you can access to get stats on demand.
                   Specify 0 to choose random available port, -1 to disable, which is default.
  strong         - Don't let GC auto-close leaking file descriptors.
  listener=S     - Specify the fully qualified name of ActivityListener class to activate from beginning.
  dumpatshutdown - Dump open file handles at shutdown.
  excludes=FILE  - Ignore files opened directly/indirectly in specific methods.
                   File lists 'some.pkg.ClassName.methodName' patterns.

== Analyzing reported stacktraces

If run on a large application with option "dumpatshutdown", there sometimes are a huge number of reported stacktraces, many of them duplicates.

In order to help with this, there is a tool https://github.com/centic9/file-leak-postprocess[file-leak-postprocess] which can be used to post-process output from file-leak-detector. It will de-duplicate stacktraces and removes uninteresting parts from the stacktraces.

== Implementation details

This project uses the JVM's support for instrumenting Java classes during startup.

It adds code to various places where files or sockets are opened and closed to print out which file descriptors have not been closed correctly.

== stack shape inconsistent error

Since this project modifies core java bytecode it requires the -Xverify:none argument when running the agent for some versions of the JVM (OpenJ9 version 1.8.x for example). See https://github.com/jenkinsci/lib-file-leak-detector/issues/37[#37] and https://github.com/jenkinsci/lib-file-leak-detector/pull/50#issue-602359846[#50] for details.

== Documentation

== Contributing

Refer to our https://github.com/jenkinsci/.github/blob/master/CONTRIBUTING.md[contribution guidelines].

To build the package:

[source,sh]

$ mvn package

The resulting package will be at file-leak-detector-${VERSION}-SNAPSHOT-jar-with-dependencies.jar.

To run integration tests:

[source,sh]

$ mvn verify

This will run the tests in the org.kohsuke.file_leak_detector.instrumented package, which are executed with instrumentation via the Java agent being active.