search

cardano-foundation / cf-ledger-sync

An open-source Java application designed to facilitate access to indexed Cardano blockchain data
Apache License 2.0
21 stars 2 forks source link

[AuxDataSerializer] Error converting metadata cbor to json #143

Closed rcmorano closed 7 months ago

rcmorano commented 7 months ago

Aloha team,

I've come across with this issue on sanchonet and preprod networks (logs are from preprod).

I'm running commit 853641d from PR #121.

[0]

2024-03-11 13:12:20.406 ERROR 1 --- [ntLoopGroup-5-1] c.b.c.y.c.m.s.AuxDataSerializer          : Error converting metadata cbor to json

com.bloxbean.cardano.client.metadata.exception.MetadataDeSerializationException: Deserialization error
        at com.bloxbean.cardano.client.metadata.helper.MetadataToJsonNoSchemaConverter.cborBytesToJson(MetadataToJsonNoSchemaConverter.java:30)
        at com.bloxbean.cardano.yaci.core.model.serializers.AuxDataSerializer.deserializeDI(AuxDataSerializer.java:69)
        at com.bloxbean.cardano.yaci.core.model.serializers.BlockSerializer.deserializeBlock(BlockSerializer.java:93)
        at com.bloxbean.cardano.yaci.core.model.serializers.BlockSerializer.deserialize(BlockSerializer.java:33)
        at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.onReceiveBlocks(BlockfetchAgent.java:128)
        at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.processResponse(BlockfetchAgent.java:83)
        at com.bloxbean.cardano.yaci.core.protocol.Agent.receiveResponse(Agent.java:44)
        at com.bloxbean.cardano.yaci.core.network.handlers.MiniProtoClientInboundHandler.channelRead(MiniProtoClientInboundHandler.java:39)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: com.bloxbean.cardano.client.metadata.exception.MetadataDeSerializationException: Deserialization error
        at com.bloxbean.cardano.client.metadata.helper.MetadataToJsonNoSchemaConverter.cborHexToJson(MetadataToJsonNoSchemaConverter.java:45)
        at com.bloxbean.cardano.client.metadata.helper.MetadataToJsonNoSchemaConverter.cborBytesToJson(MetadataToJsonNoSchemaConverter.java:28)
        ... 28 common frames omitted
Caused by: com.bloxbean.cardano.client.metadata.exception.MetadataDeSerializationException: Invalid key type : ARRAY
        at com.bloxbean.cardano.client.metadata.helper.MetadataToJsonNoSchemaConverter.processKey(MetadataToJsonNoSchemaConverter.java:95)
        at com.bloxbean.cardano.client.metadata.helper.MetadataToJsonNoSchemaConverter.processMap(MetadataToJsonNoSchemaConverter.java:76)
        at com.bloxbean.cardano.client.metadata.helper.MetadataToJsonNoSchemaConverter.processValue(MetadataToJsonNoSchemaConverter.java:110)
        at com.bloxbean.cardano.client.metadata.helper.MetadataToJsonNoSchemaConverter.processMap(MetadataToJsonNoSchemaConverter.java:77)
        at com.bloxbean.cardano.client.metadata.helper.MetadataToJsonNoSchemaConverter.cborHexToJavaMap(MetadataToJsonNoSchemaConverter.java:64)
        at com.bloxbean.cardano.client.metadata.helper.MetadataToJsonNoSchemaConverter.cborHexToJson(MetadataToJsonNoSchemaConverter.java:41)
        ... 29 common frames omitted

2024-03-11 13:12:20.434  INFO 1 --- [ntLoopGroup-5-1] c.b.c.y.s.c.service.BlockFetchService    : Publish EpochChangeEvent >>>
2024-03-11 13:12:20.435  INFO 1 --- [ntLoopGroup-5-1] c.b.c.y.s.c.service.BlockFetchService    : Processing first block of the epoch 129, block 2024948
2024-03-11 13:12:20.463  INFO 1 --- [ntLoopGroup-5-1] o.c.l.listeners.BlockEventListener       : Block  number 2024948, slot_no 54478325, hash 2cbc2a95ed9ab49b10d9016cc0201c19749c79ba28c9b55ff70860e65d67d077
2024-03-11 13:12:20.470  INFO 1 --- [ntLoopGroup-5-1] o.c.l.s.impl.block.BlockSyncServiceImpl  : Commit from block 2024948 to block 2024948
2024-03-11 13:12:20.870 ERROR 1 --- [ntLoopGroup-5-1] o.c.l.s.impl.TxMetaDataServiceImpl       : Tx hash: 42680df36ee51b932d2fd62c736b34ca6f2ecfe702e687529b565e157c335088, meta data json: null, mess: argument "content" is null
2024-03-11 13:12:20.876 ERROR 1 --- [ntLoopGroup-5-1] o.c.l.listeners.BlockEventListener       : null
java.lang.IllegalStateException
        at org.cardanofoundation.ledgersync.service.impl.TxMetaDataServiceImpl.handleAuxiliaryData(TxMetaDataServiceImpl.java:99)
        at org.cardanofoundation.ledgersync.service.impl.TxMetaDataServiceImpl.lambda$handleAuxiliaryDataMaps$1(TxMetaDataServiceImpl.java:56)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
        at org.cardanofoundation.ledgersync.service.impl.TxMetaDataServiceImpl.lambda$handleAuxiliaryDataMaps$2(TxMetaDataServiceImpl.java:58)
        at java.base/java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:833)
        at org.cardanofoundation.ledgersync.service.impl.TxMetaDataServiceImpl.handleAuxiliaryDataMaps(TxMetaDataServiceImpl.java:43)
        at org.cardanofoundation.ledgersync.service.impl.TransactionServiceImpl.handleTxContents(TransactionServiceImpl.java:204)
        at org.cardanofoundation.ledgersync.service.impl.TransactionServiceImpl.handleTxs(TransactionServiceImpl.java:146)
        at org.cardanofoundation.ledgersync.service.impl.TransactionServiceImpl.prepareAndHandleTxs(TransactionServiceImpl.java:135)
        at org.cardanofoundation.ledgersync.service.impl.block.BlockSyncServiceImpl.handleBlockSync(BlockSyncServiceImpl.java:78)
        at org.cardanofoundation.ledgersync.service.impl.block.BlockSyncServiceImpl.startBlockSyncing(BlockSyncServiceImpl.java:50)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
        at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:227)
        at io.micrometer.core.aop.TimedAspect.perform(TimedAspect.java:202)
        at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:194)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:637)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:627)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:71)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
        at org.cardanofoundation.ledgersync.service.impl.block.BlockSyncServiceImpl$$SpringCGLIB$$0.startBlockSyncing(<generated>)
        at org.cardanofoundation.ledgersync.listeners.BlockEventListener.handleAggregateBlock(BlockEventListener.java:168)
        at org.cardanofoundation.ledgersync.listeners.BlockEventListener.handleBlockEvent(BlockEventListener.java:61)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
        at org.cardanofoundation.ledgersync.listeners.BlockEventListener$$SpringCGLIB$$0.handleBlockEvent(<generated>)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:348)
        at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:233)
        at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:165)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:174)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:167)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:145)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:437)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:383)
        at com.bloxbean.cardano.yaci.store.core.service.publisher.ShelleyBlockEventPublisher.processBlockSingleThread(ShelleyBlockEventPublisher.java:175)
        at com.bloxbean.cardano.yaci.store.core.service.publisher.ShelleyBlockEventPublisher.publishBlockEvents(ShelleyBlockEventPublisher.java:54)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:391)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
        at com.bloxbean.cardano.yaci.store.core.service.publisher.ShelleyBlockEventPublisher$$SpringCGLIB$$0.publishBlockEvents(<generated>)
        at com.bloxbean.cardano.yaci.store.core.service.BlockFetchService.onBlock(BlockFetchService.java:107)
        at com.bloxbean.cardano.yaci.helper.listener.BlockFetchAgentListenerAdapter.blockFound(BlockFetchAgentListenerAdapter.java:62)
        at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.lambda$onReceiveBlocks$4(BlockfetchAgent.java:134)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
        at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.onReceiveBlocks(BlockfetchAgent.java:134)
        at com.bloxbean.cardano.yaci.core.protocol.blockfetch.BlockfetchAgent.processResponse(BlockfetchAgent.java:83)
        at com.bloxbean.cardano.yaci.core.protocol.Agent.receiveResponse(Agent.java:44)
        at com.bloxbean.cardano.yaci.core.network.handlers.MiniProtoClientInboundHandler.channelRead(MiniProtoClientInboundHandler.java:39)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:1583)
2024-03-11 13:12:20.895  INFO 1 --- [ionShutdownHook] c.b.c.yaci.core.network.NodeClient       : Shutdown connection !!!
2024-03-11 13:12:20.895  INFO 1 --- [ionShutdownHook] c.b.cardano.yaci.core.network.Session    : Disposing the session !!!
2024-03-11 13:12:20.896  INFO 1 --- [ntLoopGroup-3-1] c.b.c.yaci.core.network.NodeClient       : Connection closed !!!
2024-03-11 13:12:20.915  INFO 1 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2024-03-11 13:12:20.919  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : explorer-consumer-pool - Shutdown initiated...
2024-03-11 13:12:20.928  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : explorer-consumer-pool - Shutdown completed.
satran004 commented 7 months ago

Thanks @rcmorano . It seems the metadata has a key of array type. The current converter from Metadata to JSON doesn't expect ARRAY or MAP in the key.

We will update the converter to support ARRAY and MAP in key.

Here's the parsed cbor

{1: {[{"github": "https://github.com/mrtnetwork/On_chain"}]: "MRT"}}