search

ltearno / pom-explorer

A tool to aid managing lots of maven projects
MIT License
371 stars 65 forks source link

java.nio.file.FileSystemException: Too many open files in system #19

Open guitargice opened 8 years ago

guitargice commented 8 years ago

I get this exception when scanning through a folder containing around 50 projects, each of which containing on average 3 sub-poms.

ltearno commented 8 years ago

Thank you I will look at that. I knew already that there is a leak of file handles in the project, but your feedback confirms it. Although it's weird that it happens so early because i tested it on my machine to scan my entire maven repository ! Maybe you can send me more informations like the call stack and your machine and java config ? Thanks !

Le lun. 22 août 2016 à 21:41, guitargice notifications@github.com a écrit :

I get this exception when scanning through a folder containing around 50 projects, each of which containing on average 3 sub-poms.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ltearno/pom-explorer/issues/19, or mute the thread https://github.com/notifications/unsubscribe-auth/AALqXfOwLArsqhlHDpEgKwQkOdhTjMKyks5qiftSgaJpZM4JqOj- .

guitargice commented 8 years ago

Hi Arnaud, shame on me for not including the stack trace:

java.nio.file.FileSystemException: /users/g_uys/Documents/dev/res_mgr: Too many open files in system at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:407) at java.nio.file.Files.newDirectoryStream(Files.java:589) at fr.lteconsulting.pomexplorer.PomAnalyzer.scanPomFiles(PomAnalyzer.java:179) at fr.lteconsulting.pomexplorer.PomAnalyzer.lambda$scanPomFiles$50(PomAnalyzer.java:182) at fr.lteconsulting.pomexplorer.PomAnalyzer$$Lambda$11/1995619556.accept(Unknown Source) at java.lang.Iterable.forEach(Iterable.java:75) at fr.lteconsulting.pomexplorer.PomAnalyzer.scanPomFiles(PomAnalyzer.java:182) at fr.lteconsulting.pomexplorer.PomAnalyzer.analyze(PomAnalyzer.java:38) at fr.lteconsulting.pomexplorer.commands.AnalyzeCommand.directory(AnalyzeCommand.java:17) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at fr.lteconsulting.pomexplorer.commands.Commands.takeCommand(Commands.java:248) at fr.lteconsulting.pomexplorer.AppFactory$1.onWebsocketMessage(AppFactory.java:169) at fr.lteconsulting.pomexplorer.webserver.WebServer$3$1.lambda$onFullTextMessage$55(WebServer.java:190) at fr.lteconsulting.pomexplorer.webserver.WebServer$3$1$$Lambda$7/1677726021.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

java version "1.8.0_45" Java(TM) SE Runtime Environment (build 1.8.0_45-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

Machine is a 2015 MacBook Pro running OS X El Capitan v.10.11.6, 16GB RAM

CR1AT0RS commented 8 years ago

@ltearno @guitargice Same here: I am struggling with the same error. I changed my file open limit too. Still crashes.. ------------------ulimit -a -------------------------------------------------------------------- File size limit on machine: core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 524288 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 2048 virtual memory (kbytes, -v) unlimited ---------------------------------------------stack trace ----------------------------------

ava.nio.file.FileSystemException: /Users/xxxuser/blt/app/main/core/yd: Too many open files at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:407) at java.nio.file.Files.newDirectoryStream(Files.java:589) at fr.lteconsulting.pomexplorer.PomAnalyzer.scanPomFiles(PomAnalyzer.java:191) at fr.lteconsulting.pomexplorer.PomAnalyzer.lambda$scanPomFiles$3(PomAnalyzer.java:194) at java.lang.Iterable.forEach(Iterable.java:75) at fr.lteconsulting.pomexplorer.PomAnalyzer.scanPomFiles(PomAnalyzer.java:194) at fr.lteconsulting.pomexplorer.PomAnalyzer.analyze(PomAnalyzer.java:40) at fr.lteconsulting.pomexplorer.commands.AnalyzeCommand.directory(AnalyzeCommand.java:28) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at fr.lteconsulting.pomexplorer.commands.Commands.takeCommand(Commands.java:248) at fr.lteconsulting.pomexplorer.AppFactory$1.onWebsocketMessage(AppFactory.java:169) at fr.lteconsulting.pomexplorer.webserver.WebServer$3$1.lambda$onFullTextMessage$0(WebServer.java:190) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

ltearno commented 8 years ago

Hi, The commit 2ebbb362509e9d25ccdee5220aa412b35c6630ff (last one) should fix your problem. Although i am not completely sure because i didn't manage to reproduce the bug (pom explorer parses in 5 seconds the 2500 pom files of my maven repo), the fix should make the program gentler with the system...

Could you please confirm that it works now ? Thanks