impossibl / pgjdbc-ng

A new JDBC driver for PostgreSQL aimed at supporting the advanced features of JDBC and Postgres
https://impossibl.github.io/pgjdbc-ng
Other
596 stars 108 forks source link

LEAK: ByteBuf.release() was not called before it's garbage-collected #572

Closed andreak closed 2 years ago

andreak commented 2 years ago

Hi all, I'm saving an JPA @Entity using EclipseLink and pgjdbc-ng-0.8.9 against PG-14.2, and I get this stacktrace:

2022.03.10 18:32:49:793 [PG-JDBC I/O (1)] [TTUF] [] [] ERROR com.impossibl.shadow.io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
        com.impossibl.shadow.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:402)
        com.impossibl.shadow.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
        com.impossibl.shadow.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
        com.impossibl.shadow.io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
        com.impossibl.shadow.io.netty.buffer.ByteBufUtil.writeUtf8(ByteBufUtil.java:623)
        com.impossibl.postgres.jdbc.PGBuffersArray.encode(PGBuffersArray.java:126)
        com.impossibl.postgres.jdbc.PGBuffersArray.encode(PGBuffersArray.java:110)
        com.impossibl.postgres.jdbc.PGBuffersArray.encode(PGBuffersArray.java:92)
        com.impossibl.postgres.jdbc.PGDirectConnection.createArrayOf(PGDirectConnection.java:1230)
        com.zaxxer.hikari.pool.HikariProxyConnection.createArrayOf(HikariProxyConnection.java)
        org.eclipse.persistence.internal.databaseaccess.DatabasePlatform.createArray(DatabasePlatform.java:3444)
        org.eclipse.persistence.internal.databaseaccess.DatabasePlatform.createArray(DatabasePlatform.java:3415)
        org.eclipse.persistence.mappings.structures.ObjectRelationalDataTypeDescriptor.buildFieldValueFromDirectValues(ObjectRelationalDataTypeDescriptor.java:112)
        org.eclipse.persistence.mappings.foundation.AbstractCompositeDirectCollectionMapping.writeFromObjectIntoRow(AbstractCompositeDirectCollectionMapping.java:625)
        org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildRow(ObjectBuilder.java:1608)
        org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildRow(ObjectBuilder.java:1596)
        org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.insertObjectForWrite(DatabaseQueryMechanism.java:466)
        org.eclipse.persistence.queries.InsertObjectQuery.executeCommit(InsertObjectQuery.java:82)
        org.eclipse.persistence.queries.InsertObjectQuery.executeCommitWithChangeSet(InsertObjectQuery.java:92)
        org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.executeWriteWithChangeSet(DatabaseQueryMechanism.java:316)
        org.eclipse.persistence.queries.WriteObjectQuery.executeDatabaseQuery(WriteObjectQuery.java:60)
        org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:911)
        org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:810)
        org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWorkObjectLevelModifyQuery(ObjectLevelModifyQuery.java:110)
        org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWork(ObjectLevelModifyQuery.java:87)
        org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:3004)
        org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1898)
        org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1880)
        org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1830)
        org.eclipse.persistence.internal.sessions.CommitManager.commitNewObjectsForClassWithChangeSet(CommitManager.java:229)
        org.eclipse.persistence.internal.sessions.CommitManager.commitAllObjectsWithChangeSet(CommitManager.java:128)
        org.eclipse.persistence.internal.sessions.AbstractSession.writeAllObjectsWithChangeSet(AbstractSession.java:4409)
        org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabase(UnitOfWorkImpl.java:1503)
        org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabaseWithPreBuiltChangeSet(UnitOfWorkImpl.java:1649)
        org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.writeChanges(RepeatableWriteUnitOfWork.java:457)
        org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:975)
        jdk.internal.reflect.GeneratedMethodAccessor339.invoke(Unknown Source)
        java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        java.base/java.lang.reflect.Method.invoke(Method.java:567)
        org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:311)
        jdk.proxy2/jdk.proxy2.$Proxy76.flush(Unknown Source)
        org.scala_libs.jpa.ScalaEntityManager.flush(ScalaEntityManager.scala:174)
        org.scala_libs.jpa.ScalaEntityManager.flush$(ScalaEntityManager.scala:174)
        no.officenet.origo.core.infrastructure.jpa.OrigoScalaEntityManager.flush(RepositorySupport.scala:33)
        no.officenet.origo.core.infrastructure.jpa.RepositorySupport.flush(RepositorySupport.scala:25)
        no.officenet.origo.core.infrastructure.jpa.RepositorySupport.flush$(RepositorySupport.scala:23)
        no.officenet.origo.core.domain.service.calendar.repository.CalendarAttendeeRepositoryImpl.flush(CalendarAttendeeRepository.scala:19)
        jdk.internal.reflect.GeneratedMethodAccessor338.invoke(Unknown Source)
        java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        java.base/java.lang.reflect.Method.invoke(Method.java:567)
        org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        jdk.proxy2/jdk.proxy2.$Proxy186.flush(Unknown Source)
        no.officenet.origo.calendar.application.service.calendar.CalendarEntryAppService.create(CalendarEntryAppService.scala:85)
        no.officenet.origo.calendar.application.service.calendar.CalendarEntryAppService.create$(CalendarEntryAppService.scala:79)
        no.officenet.origo.calendar.application.service.calendar.CalendarEntryAppServiceImpl.create(CalendarEntryAppService.scala:636)
        no.officenet.origo.calendar.application.service.calendar.CalendarEntryAppService.$anonfun$createNewInvitationEntryFromIcal$1(CalendarEntryAppService.scala:452)

Other places where we extract data from SQLArray we've wrapped them in this method (in Scala, but you get the idea):

    private def getArrayValue[T](sqlArray: sql.Array): List[T] = {
        if (sqlArray != null) {
            val ret = sqlArray.getArray(SQLUtils.typeMap).asInstanceOf[Array[T]]
            sqlArray.free()
            ret.toList
        } else {
            Nil
        }
    }

Can this LEAK be caused by EclipseLink not calling sqlArray.free() or is it an actual leak in the driver, or something else?

andreak commented 2 years ago

It seems it's EclipseLink not free()-ing the Array after creation. According to https://docs.oracle.com/javase/tutorial/jdbc/basics/array.html the Array returned should be freed after use. The offending code is in EL's AbstractCompositeDirectCollectionMapping.writeFromObjectIntoRow where the result-value of this.getDescriptor().buildFieldValueFromDirectValues is never freed:

@Override
public void writeFromObjectIntoRow(Object object, AbstractRecord row, AbstractSession session, WriteType writeType) {
    if (this.isReadOnly()) {
        return;
    }

    Object attributeValue = this.getAttributeValueFromObject(object);
    if (attributeValue == null) {
        row.put(this.getField(), null);
        return;
    }

    ContainerPolicy cp = this.getContainerPolicy();

    Vector elements = new Vector(cp.sizeFor(attributeValue));
    for (Object iter = cp.iteratorFor(attributeValue); cp.hasNext(iter);) {
        Object element = cp.next(iter, session);
        if (this.getValueConverter() != null) {
            element = getValueConverter().convertObjectValueToDataValue(element, session);
        }
        if (element != null) {
            elements.addElement(element);
        }
    }

    Object fieldValue = null;
    if (!elements.isEmpty()) {
        fieldValue = this.getDescriptor().buildFieldValueFromDirectValues(elements, elementDataTypeName, session);
    }
    row.put(this.getField(), fieldValue);
    // fieldValue should be free() here.