Closed volosied closed 8 months ago
I see quite a bit changed with respect to the inference in Java 1.8. So, it's possible we create few more objects since 1.8. But three times the heap consumption doesn't sound right.
Although the stack doesn't include, I suspect the extra type bindings are created by this code - org.eclipse.jdt.internal.compiler.lookup.Scope.Substitutor.substitute(Substitution, TypeBinding)
Copying @stephan-herrmann and @srikanth-sankaran who may have some idea about this part of the compiler.
@volosied Could you please share a reproducible code snippet here?
Unfortunately, I can't gather more at this time. However, we found a workaround by compiling in batches rather than all at once.
Feel free to close this ticket, and thanks for your assistance!
What do we know?
TypeSystem
instance is root of the memory problem (it's not an AnnotatedTypeSystem
, so this is not about annotations).Here's a theory: during type inference we may need to create intermediate ParameterizedTypeBinding
s, which all are stored in TypeSystem
. Normally, such storing is the correct thing to do, to ensure identity of type bindings. But when during type inference some type arguments are InferenceVariable
s or just contain such at any nesting level, then the resulting ParameterizedTypeBinding
may cause some form a memory leak: semantically, the life span of any inference variable is just the length of resolving the outer-most expression involved. But by storing in TypeSystem
all those inference variables will survive for the entire compiler invocation (i.e., until the pair of LookupEnvironment
and TypeSystem
will be freed). In fact not just the inference variable, but the enclosing parameterized type binding is wasted, too. And they might transitively keep more objects alive.
@srikanth-sankaran does the master of TypeSystem
see a way to clean up the cache at the end of (outermost) type inference, without creating a new disproportionate performance penalty (cpu-wise)? During experiments, I found that in huge generic types even asking isProperType()
may have significant cost, ups.
Should inference itself remember all non-proper parameterized type bindings created during its course, perhaps each inference variable could keep track of those derived bindings, for easy deallocation?
Or would this create new risks in TypeSystem when after freeing we re-use a previously used slot?
Btw, this stack frame is not JDT:
at org/eclipse/jdt/internal/compiler/ast/_expression_.resolve(_expression_.java:1113(Compiled Code))
Should we worry about that?
I wonder, how exactly com/ibm/ws/jsp/translator/compiler/JDTCompiler
relates to ecj.
@jukzi jukzi closed this as not planned Feb 14, 2024
Before this goes into neverland let me ask @srikanth-sankaran : have you seen my theory around TypeSystem holding types that are strictly obsolete after the inference session that created them? I believe we could play with those mechanisms even without a strict reproducer plus heap measurements?
I'll take a look.
Originally send an email here: https://www.eclipse.org/lists/jdt-dev/msg02297.html
Version: ECJ 4.20 (We cannot test any newer ECJ releases since our application and server run on Java 8)
Problem: The issue is that the 1.8 source/target compiler option utilizes a lot more memory than the 1.7 option when compiling translated JSPs.
1.7 requires 1GB of heap while 1.8 requires 5 GB. Here's a snippet of the heapdump:
The stacktrace for Thread-173 is:
I can provide more information if needed. This behavior doesn't seem right, but perhaps it is normal based on the new Java 8 language features? I would appreciate if someone could provide an explanation -- I don't have the knowledge to figure this out myself.
Thank you!