[BUG] Long compilation invocations with Eclipse Code Mining feature

[hheg]

Describe the bug A clear and concise description of what the bug is.

Running with lombok and Eclipse's CodeMining features creates high and long time running loads where it seems like it's trying to traverse the CU's over and over again. If you are fast enough (which isn't hard to be in this scenario), CodeMining launches more and more "processing" threads hogging up all cores, which seems to never end unless you leave the machine for a couple of hours. Disabling CodeMining removes/fixes this behavior.

To Reproduce Steps to reproduce the behavior, preferably in the form of the smallest java source file you can make that will show the problem when compiled with javac -cp lombok.jar ExampleFile.java or as sole java file in a fresh new eclipse project.

If this is not possible, give us enough to reproduce the problem. If you have stack traces or error messages please include all of them, and include screen shots if that will help explain the problem. thread.txt

Expected behavior A clear and concise description of what you expected to happen. It shouldn't degrade and CPU's shouldn't be at 100%

Version info (please complete the following information):

• Lombok version 1.18.10

• Platform (javac or eclipse, and if so, what is the output of javac -version / the version listed in the about... dialog of eclipse. javac --version javac 13.0.2 Eclipse 4.15

Additional context Add any other context about the problem here. Reported to Eclipse but closed because they thought this was something that Lombok was doing. https://bugs.eclipse.org/bugs/show_bug.cgi?id=560139

[randakar]

Why do you think this is a problem with lombok, not codemining? Or have you filed a bug with eclipse as well?

On Fri, Apr 17, 2020, 18:19 Henrik Hegardt notifications@github.com wrote:

Describe the bug A clear and concise description of what the bug is.

Running with lombok and Eclipse's CodeMining features creates high and long time running loads where it seems like it's trying to traverse the CU's over and over again. If you are fast enough (which isn't hard to be in this scenario), CodeMining launches more and more "processing" threads hogging up all cores, which seems to never end unless you leave the machine for a couple of hours. Disabling CodeMining removes/fixes this behavior.

To Reproduce Steps to reproduce the behavior, preferably in the form of the smallest java source file you can make that will show the problem when compiled with javac -cp lombok.jar ExampleFile.java or as sole java file in a fresh new eclipse project.

If this is not possible, give us enough to reproduce the problem. If you have stack traces or error messages please include all of them, and include screen shots if that will help explain the problem. thread.txt https://github.com/rzwitserloot/lombok/files/4494016/thread.txt

Expected behavior A clear and concise description of what you expected to happen. It shouldn't degrade and CPU's shouldn't be at 100%

Version info (please complete the following information):

-

Lombok version

Platform (javac or eclipse, and if so, what is the output of javac -version / the version listed in the about... dialog of eclipse. javac --version javac 13.0.2 Eclipse 4.15

Additional context Add any other context about the problem here. Reported to Eclipse but closed because they thought this was something that Lombok was doing. https://bugs.eclipse.org/bugs/show_bug.cgi?id=560139

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rzwitserloot/lombok/issues/2430, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABIERLXVK5HVOV2JW3RZD3RNB6RVANCNFSM4MK332PA .

[randakar]

Looking into threads.txt it seems codemining is somehow causing lombok to run a bunch of times in parallel. And some threads are blocked, trying to traverse directories for lombok.config:

"ForkJoinPool.commonPool-worker-7" #947 daemon prio=1 os_prio=0 cpu=21306,01ms elapsed=303,46s tid=0x00007fdf68126000 nid=0x1729 waiting for monitor entry [0x00007fdf8a2b3000] java.lang.Thread.State: BLOCKED (on object monitor) at lombok.core.configuration.FileSystemSourceCache.getSourceForDirectory(FileSystemSourceCache.java:156)

    - locked <0x00000000a7317f10> (a

lombok.core.configuration.FileSystemSourceCache$Content)

    at lombok.core.configuration.FileSystemSourceCache$2$1.findNext(FileSystemSourceCache.java:132)
    at lombok.core.configuration.FileSystemSourceCache$2$1.hasNext(FileSystemSourceCache.java:118)
    at lombok.core.configuration.BubblingConfigurationResolver.resolve(BubblingConfigurationResolver.java:44)
    at lombok.core.LombokConfiguration.read(LombokConfiguration.java:71)

Does this project have a lombok.config file? Does it help to configure one at the root level with "config.stopBubbling = true" set in there?

See https://projectlombok.org/features/configuration for how this works.

On Sat, Apr 18, 2020, 08:15 Floris Kraak randakar@gmail.com wrote:

Why do you think this is a problem with lombok, not codemining? Or have you filed a bug with eclipse as well?

On Fri, Apr 17, 2020, 18:19 Henrik Hegardt notifications@github.com wrote:

Describe the bug A clear and concise description of what the bug is.

Running with lombok and Eclipse's CodeMining features creates high and long time running loads where it seems like it's trying to traverse the CU's over and over again. If you are fast enough (which isn't hard to be in this scenario), CodeMining launches more and more "processing" threads hogging up all cores, which seems to never end unless you leave the machine for a couple of hours. Disabling CodeMining removes/fixes this behavior.

To Reproduce Steps to reproduce the behavior, preferably in the form of the smallest java source file you can make that will show the problem when compiled with javac -cp lombok.jar ExampleFile.java or as sole java file in a fresh new eclipse project.

If this is not possible, give us enough to reproduce the problem. If you have stack traces or error messages please include all of them, and include screen shots if that will help explain the problem. thread.txt https://github.com/rzwitserloot/lombok/files/4494016/thread.txt

Expected behavior A clear and concise description of what you expected to happen. It shouldn't degrade and CPU's shouldn't be at 100%

Version info (please complete the following information):

-

Lombok version

Platform (javac or eclipse, and if so, what is the output of javac -version / the version listed in the about... dialog of eclipse. javac --version javac 13.0.2 Eclipse 4.15

Additional context Add any other context about the problem here. Reported to Eclipse but closed because they thought this was something that Lombok was doing. https://bugs.eclipse.org/bugs/show_bug.cgi?id=560139

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rzwitserloot/lombok/issues/2430, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABIERLXVK5HVOV2JW3RZD3RNB6RVANCNFSM4MK332PA .

[Rawi01]

I enabled code mining for a small test project and did not notice any differences. There are some weird interactions with the lombok annotations e.g. parameter names shown above the class name for delegates but this seems unrelated to this issue. @hheg have you tried it with a small project too? can you share an example project? @randakar I also noticed the config file reads, there is another performance related issue (#1717) that also mentions the config file access.

[hheg]

Why do you think this is a problem with lombok, not codemining? Or have you filed a bug with eclipse as well? …

Reported to Eclipse but closed because they thought this was something that Lombok was doing. https://bugs.eclipse.org/bugs/show_bug.cgi?id=560139

[hheg]

I enabled code mining for a small test project and did not notice any differences. There are some weird interactions with the lombok annotations e.g. parameter names shown above the class name for delegates but this seems unrelated to this issue. @hheg have you tried it with a small project too? can you share an example project? @randakar I also noticed the config file reads, there is another performance related issue (#1717) that also mentions the config file access.

No, I haven't tried with a small project, but I believe that since if you leave the project/eclipse for a few hours the processing completes. To me that seems to indicate that it would/could be present in a small project, but it would still complete so fast you couldn't tell.

[hheg]

Does this project have a lombok.config file? Does it help to configure one at the root level with "config.stopBubbling = true" set in there? See https://projectlombok.org/features/configuration for how this works.

No it doesn't.

[randakar]

Pity, it would have pointed at that file cache quite clearly if it did.

Given that we do not have a public reproduction path we are going to have a hard time debugging this.

Have you ever ran an application with something like VisualVM ( https://docs.oracle.com/javase/8/docs/technotes/guides/visualvm/profiler.html ) attached?

I know it's a hassle but it may be the only way to figure out what is going on.

On Sat, Apr 18, 2020, 11:08 Henrik Hegardt notifications@github.com wrote:

Does this project have a lombok.config file? Does it help to configure one at the root level with "config.stopBubbling = true" set in there? See https://projectlombok.org/features/configuration for how this works.

No it doesn't.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rzwitserloot/lombok/issues/2430#issuecomment-615808181, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABIERMTH66JGJSMUU2EHHTRNFUZ3ANCNFSM4MK332PA .

[randakar]

Wait, did you mean it doesn't have a config file, or did you mean that adding one doesn't help?

On Sat, Apr 18, 2020, 11:50 Floris Kraak randakar@gmail.com wrote:

Pity, it would have pointed at that file cache quite clearly if it did.

Given that we do not have a public reproduction path we are going to have a hard time debugging this.

Have you ever ran an application with something like VisualVM ( https://docs.oracle.com/javase/8/docs/technotes/guides/visualvm/profiler.html ) attached?

I know it's a hassle but it may be the only way to figure out what is going on.

On Sat, Apr 18, 2020, 11:08 Henrik Hegardt notifications@github.com wrote:

Does this project have a lombok.config file? Does it help to configure one at the root level with "config.stopBubbling = true" set in there? See https://projectlombok.org/features/configuration for how this works.

No it doesn't.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rzwitserloot/lombok/issues/2430#issuecomment-615808181, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABIERMTH66JGJSMUU2EHHTRNFUZ3ANCNFSM4MK332PA .

[Rawi01]

I tried a bigger project (10 minutes compile time) and there is still no difference in compile times with or without code mining.

While fiddling around I noticed a bunch of interesting things:

• Code mining doesn't show a progress bar...
• It only calculate stuff for visible parts of a file
• Object methods (e.g. toString(), equals(), ...) reference count is really high and takes a few seconds to calculate. Seems like it counts every invocation of Object.toString() in all dependencies. Thats eactly how the normal reference search in eclipse works.

It seems like code mining for these common methods is quite slow. Using something like @Data adds multiple of these slow methods in the same line and triggers a lot of calculations if you open such a file. It might be possible to disable code mining for lombok generated methods.

For my test I used /lombok/test/transform/resource/before/DataPlain.java which needs ~2 minutes to complete code mining. It's the same if I use the delombok version and scroll to the end (to trigger code mining for all methods)

@hheg can you try to disable reference counting and check if the problem is still there?

[randakar]

Good, so we do have a reproduction path now.

On Sat, Apr 18, 2020, 16:02 Rawi01 notifications@github.com wrote:

I tried a bigger project (10 minutes compile time) and there is still no difference in compile times with or without code mining.

While fiddling around I noticed a bunch of interesting things:

• Code mining doesn't show a progress bar...
• It only calculate stuff for visible parts of a file
• Object methods (e.g. toString(), equals(), ...) reference count is really high and takes a few seconds to calculate. Seems like it counts every invocation of Object.toString() in all dependencies. Thats eactly how the normal reference search in eclipse works.

[image: grafik] https://user-images.githubusercontent.com/5850477/79639738-c8f39580-818d-11ea-9e67-5bfbfdbc1d11.png

It seems like code mining for these common methods is quite slow. Using something like @Data adds multiple of these slow methods in the same line and triggers a lot of calculations if you open such a file. It might be possible to disable code mining for lombok generated methods.

For my test I used /lombok/test/transform/resource/before/DataPlain.java which needs ~2 minutes to complete code mining. It's the same if I use the delombok version and scroll to the end (to trigger code mining for all methods)

@hheg https://github.com/hheg can you try to disable reference counting and check if the problem is still there?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rzwitserloot/lombok/issues/2430#issuecomment-615876881, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABIERIJ4Z45FCPITZKFSTLRNGXINANCNFSM4MK332PA .

[hheg]

Sorry that I haven't answered earlier. @Rawi01 Turning off reference counting does indeed solve the problem.

[hheg]

An observation, Code Mining seems to be lazy, meaning that it gets invoked whenever you select a file. I have a lot of @Value annotated classes. Clicking around in classes where there's no lombok code seems to be indexed fast. Though when navigating to a @Value annotated class seems to trigger this behavior.

[hheg]

Wait, did you mean it doesn't have a config file, or did you mean that adding one doesn't help?

There's none. I haven't tried with adding config.stopBubbling = true

[randakar]

Please try it then.

Given the testcase @Rawi01 found whether it will help is a bit doubtful but it would be nice to be able to rule things out.

One step someone can take is to run either testcase under VisualVM and take a look at what the threads are doing under the hood. If some of them block (like that one that blocked on the file cache) that may point at an issue, depending a bit on how long and how often threads end up blocking.

Given however that the delomboked code seems to be having the same issue someone should come back to https://bugs.eclipse.org/bugs/show_bug.cgi?id=560139 and poke them with the results running codemining on the delomboked class. Since that delomboked class is perfectly good java they should not be trying to blame lombok for this.

[randakar]

For my test I used /lombok/test/transform/resource/before/DataPlain.java which needs ~2 minutes to complete code mining. It's the same if I use the delombok version and scroll to the end (to trigger code mining for all methods)

That one. Unless that delomboked code does something crazy (doubtful) this does not sound like a real lombok issue.

[hheg]

Wait, did you mean it doesn't have a config file, or did you mean that adding one doesn't help?

There's none. I haven't tried with adding config.stopBubbling = true

I added one. Problem still persist.

[randakar]

As I kind of expected, but it's nice to be certain.

[Rawi01]

I think this is a general search performance problem in eclipse. Just pick a class in a bigger project, implement an equals() or toString() method (thats what lombok @Value does), right click on that method and search for references in your workspace. For a small project this takes 5-10 seconds per method, for a bigger project it takes multiple minutes (for me). Thats exactly what happens if code mining counts references. If this task continues after a file gets closed...

To search the method references code mining parses the code, thats why lombok is part of the stack trace

[randakar]

Yeah, feels like that to me too.

Doesn't make it a lombok problem at all, does it? Has anyone provided the eclipse folks with feedback yet?

On Mon, Apr 20, 2020, 21:22 Rawi01 notifications@github.com wrote:

I think this is a general search performance problem in eclipse. Just pick a class in a bigger project, implement an equals() or toString() method (thats what lombok @Value does), right click on that method and search for references in your workspace. For a small project this takes 5-10 seconds per method, for a bigger project it takes multiple minutes (for me). Thats exactly what happens if code mining counts references. If this task continues after a file gets closed...

To search the method references code mining parses the code, thats why lombok is part of the stack trace

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rzwitserloot/lombok/issues/2430#issuecomment-616758101, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABIERID4NXNE4MMMIFFUBLRNSOIHANCNFSM4MK332PA .

[hheg]

Yeah, feels like that to me too. Doesn't make it a lombok problem at all, does it? Has anyone provided the eclipse folks with feedback yet?

I have a different similar large project with codemining turned on but without lombok and it doesn't suffer from this kind of delays. Yes the codemining is consuming some CPU to do the calculations but its nowhere near the intensity when lombok is active. I had thought to use lombok but because of this problem I haven't converted it to use lombok.

[randakar]

The only way to be sure is to simply delombok everything up front and comparing it with the lombok version.

Thing is, to some extend a slowdown is to be expected. 1) lombok needs to do work when compiling. Codemining compiles everything, possibly multiple times in parallel. 2) because lombok makes your code more compact you effectively have more code on screen at any given time, so codemining has to do more work.

The question is whether this really is due to a bug in lombok or because codemining is doing crazy things that get worse due to the circumstances above.

On Tue, Apr 21, 2020, 08:24 Henrik Hegardt notifications@github.com wrote:

Yeah, feels like that to me too. Doesn't make it a lombok problem at all, does it? Has anyone provided the eclipse folks with feedback yet?

I have a different similar large project with codemining turned on but without lombok and it doesn't suffer from this kind of delays. Yes the codemining is consuming some CPU to do the calculations but its nowhere near the intensity when lombok is active. I had thought to use lombok but because of this problem I haven't converted it to use lombok.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rzwitserloot/lombok/issues/2430#issuecomment-616978338, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABIERK4QJEJPXC4D7QERWDRNU3YJANCNFSM4MK332PA .

[randakar]

Looks like the codemining folks are on it. See activity in https://bugs.eclipse.org/bugs/show_bug.cgi?id=560139

I suggest we close this for the time being.

On Wed, Apr 22, 2020 at 9:04 AM Floris Kraak randakar@gmail.com wrote:

The only way to be sure is to simply delombok everything up front and comparing it with the lombok version.

Thing is, to some extend a slowdown is to be expected. 1) lombok needs to do work when compiling. Codemining compiles everything, possibly multiple times in parallel. 2) because lombok makes your code more compact you effectively have more code on screen at any given time, so codemining has to do more work.

The question is whether this really is due to a bug in lombok or because codemining is doing crazy things that get worse due to the circumstances above.

On Tue, Apr 21, 2020, 08:24 Henrik Hegardt notifications@github.com wrote:

Yeah, feels like that to me too. Doesn't make it a lombok problem at all, does it? Has anyone provided the eclipse folks with feedback yet?

I have a different similar large project with codemining turned on but without lombok and it doesn't suffer from this kind of delays. Yes the codemining is consuming some CPU to do the calculations but its nowhere near the intensity when lombok is active. I had thought to use lombok but because of this problem I haven't converted it to use lombok.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rzwitserloot/lombok/issues/2430#issuecomment-616978338, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABIERK4QJEJPXC4D7QERWDRNU3YJANCNFSM4MK332PA .

-- "Don't only practice your art, but force your way into it's secrets, for it and knowledge can raise men to the divine." -- Ludwig von Beethoven