looping on error while generating artwork

[jluc2808]

for some external reason i had to reconstruct my database and perform a new scan , while getting artwork , jetty is looping thru the same error (for almost 2 hours and more than 500 entries of the same one)

[27.03.2016 13:22:52] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Duplicate entry '535-tmdb-kjA2VkYx929rPKMPFQRb0lPqY3w' for key 'UIX_ARTWORKLOCATED_NATURALID' [27.03.2016 13:22:52] ERROR org.yamj.core.service.artwork.ArtworkScannerService - Failed storing artwork 535-POSTER [27.03.2016 13:22:52] WARN org.yamj.core.service.artwork.ArtworkScannerService - Storage error org.hibernate.exception.ConstraintViolationException: could not execute statement at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:59) at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:207) at org.hibernate.dialect.identity.GetGeneratedKeysDelegate.executeAndExtract(GetGeneratedKeysDelegate.java:57) at org.hibernate.id.insert.AbstractReturningDelegate.performInsert(AbstractReturningDelegate.java:42) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2827) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3398) at org.hibernate.action.internal.EntityIdentityInsertAction.execute(EntityIdentityInsertAction.java:81) at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:597) at org.hibernate.engine.spi.ActionQueue.addResolvedEntityInsertAction(ActionQueue.java:232) at org.hibernate.engine.spi.ActionQueue.addInsertAction(ActionQueue.java:213) at org.hibernate.engine.spi.ActionQueue.addAction(ActionQueue.java:256) at org.hibernate.event.internal.AbstractSaveEventListener.addInsertAction(AbstractSaveEventListener.java:318) at org.hibernate.event.internal.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:275) at org.hibernate.event.internal.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:182) at org.hibernate.event.internal.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:113) at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEventListener.java:192) at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.java:177) at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.performSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:97) at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:73) at org.hibernate.internal.SessionImpl.fireSaveOrUpdate(SessionImpl.java:651) ....... at java.lang.Thread.run(Unknown Source) Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '535-tmdb-kjA2VkYx929rPKMPFQRb0lPqY3w' for key 'UIX_ARTWORKLOCATED_NATURALID' at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) at com.mysql.jdbc.Util.getInstance(Util.java:387) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:932) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073) at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2009) at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5094) at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1994) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:204) ... 37 more

[modmax]

There seems to be a second artwork with the same hash code (kjA2VkYx929rPKMPFQRb0lPqY3w) for the online source "TMDB" (TheMovieDb) for artwork with id 535.

Without more LOG informations (which will describe the execution chain whjich leads to the error), i cannot clearly determine what kind of problem this is.

[jluc2808]

i hope this could help , find the artwork 535 and reset the status to UPDATED (because i stop the looping by setting status = ERROR) and immediatly the error restart with [29.03.2016 20:22:12] INFO org.yamj.core.scheduling.ArtworkScanScheduler - Found 1 artwork objects to scan; process with 1 threads [29.03.2016 20:22:12] DEBUG org.yamj.core.service.artwork.ArtworkScannerService - Scan online for poster: Artwork [ID=535, type=POSTER, boxedset-id='Madagascar'] [29.03.2016 20:22:12] DEBUG org.yamj.core.service.artwork.ArtworkScannerService - Use tmdb scanner for Artwork [ID=535, type=POSTER, boxedset-id='Madagascar'] [29.03.2016 20:22:12] DEBUG org.yamj.core.service.artwork.online.TheMovieDbArtworkScanner - Got 12 POSTER artworks from TMDb for id 14740 [29.03.2016 20:22:12] DEBUG org.yamj.core.service.artwork.online.TheMovieDbArtworkScanner - Found 4 POSTER artworks for TMDb id 14740 and language 'fr' [29.03.2016 20:22:12] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Duplicate entry '535-tmdb-kjA2VkYx929rPKMPFQRb0lPqY3w' for key 'UIX_ARTWORKLOCATED_NATURALID' [29.03.2016 20:22:12] ERROR org.yamj.core.service.artwork.ArtworkScannerService - Failed storing artwork 535-POSTER [29.03.2016 20:22:12] WARN org.yamj.core.service.artwork.ArtworkScannerService - Storage error org.hibernate.exception.ConstraintViolationException: could not execute statement at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:59) at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:207) at org.hibernate.dialect.identity.GetGeneratedKeysDelegate.executeAndExtract(GetGeneratedKeysDelegate.java:57) at org.hibernate.id.insert.AbstractReturningDelegate.performInsert(AbstractReturningDelegate.java:42) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2827) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3398) at org.hibernate.action.internal.EntityIdentityInsertAction.execute(EntityIdentityInsertAction.java:81) at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:597) at org.hibernate.engine.spi.ActionQueue.addResolvedEntityInsertAction(ActionQueue.java:232) at org.hibernate.engine.spi.ActionQueue.addInsertAction(ActionQueue.java:213) at org.hibernate.engine.spi.ActionQueue.addAction(ActionQueue.java:256) at org.hibernate.event.internal.AbstractSaveEventListener.addInsertAction(AbstractSaveEventListener.java:318) at org.hibernate.event.internal.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:275) at org.hibernate.event.internal.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:182) at org.hibernate.event.internal.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:113) at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEventListener.java:192) at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.java:177) at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.performSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:97) at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:73) at org.hibernate.internal.SessionImpl.fireSaveOrUpdate(SessionImpl.java:651) at org.hibernate.internal.SessionImpl.saveOrUpdate(SessionImpl.java:643) at org.hibernate.internal.SessionImpl.saveOrUpdate(SessionImpl.java:638) at org.yamj.core.hibernate.HibernateDao.storeAll(HibernateDao.java:95) at org.yamj.core.database.service.ArtworkStorageService.updateArtwork(ArtworkStorageService.java:98) at org.yamj.core.database.service.ArtworkStorageService$$FastClassBySpringCGLIB$$9e265e32.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) at org.yamj.core.database.service.ArtworkStorageService$$EnhancerBySpringCGLIB$$13701c6c.updateArtwork() at org.yamj.core.service.artwork.ArtworkScannerService.processQueueElement(ArtworkScannerService.java:153) at org.yamj.core.scheduling.QueueProcessRunner.run(QueueProcessRunner.java:46) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '535-tmdb-kjA2VkYx929rPKMPFQRb0lPqY3w' for key 'UIX_ARTWORKLOCATED_NATURALID' at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) at com.mysql.jdbc.Util.getInstance(Util.java:387) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:932) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861) at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073) at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2009) at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5094) at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1994) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:204) ... 37 more

[modmax]

Can you see if there is already a stored entry in artwork located with same source and hash code? SELECT * FROM artwork_located WHERE artwork_id=535 ?

[jluc2808]

no result with the sql SELECT * from artwork_located where artwork_id=535

[modmax]

Problem was, that the Madagascar collection contains 2 images with same url (but just different ratings). So there were unexpected duplicated in the located artworks; which will now be eliminated.

[jluc2808]

tested ok