manticoresoftware / manticoresearch-java

Official Java client for Manticore Search
MIT License
41 stars 5 forks source link

Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `com.manticoresearch.client.model.SuccessResponse` from Array value (token `JsonToken.START_ARRAY`) #20

Closed FriskKiddo closed 5 months ago

FriskKiddo commented 8 months ago

I deployed a manticoresearch service by helm(https://github.com/manticoresoftware/manticoresearch-helm/blob/master/charts/manticoresearch/values.yaml) , and the related images version is manticoresearch/helm-balancer:6.2.13.1 and manticoresearch/helm-worker:6.2.13.1. And then i want to consumer kafka message and insert into a manticore index. The follow is the ddl of the index: CREATE TABLE application ( id bigint, __project__ text, __file__ text, __host__ text, __level__ text, __raw_log__ text, content text, source_type text, __dt__ timestamp )

The following is a part of kafka consumer code: ` ApiClient defaultClient = new ApiClient(); defaultClient.setBasePath("http://xxx:9308"); IndexApi indexApi = new IndexApi(defaultClient); this.consumer.subscribe(Collections.singletonList("jy-crawl-process-zjk-application"));

    while (true) {
        ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(1000));
        for (ConsumerRecord<String, String> record : records) {
            System.out.println("MESSAGE|" + record.value());
            JSONObject jsonObject = JSONObject.parseObject(record.value());
            InsertDocumentRequest request = new InsertDocumentRequest();
            request.index("application");
            request.cluster("manticore_cluster");
            request.doc(jsonObject.getInnerMap());
            try {
                indexApi.insert(request);
            } catch (ApiException e) {
                System.out.println("error");
            }
        }
    }

`

But i got a exception as following: MESSAGE|{"__dt__":"2024-02-26 14:46:28","__file__":"/home/admin/xxx/logs/application.log","__host__":"jy-xxx.center.na610","__level__":"info","__project__":"application","__raw_log__":"2024-02-26 14:46:26.588 ERROR 3205 --- [ConsumeMessageThread_16] com.xxx.jy.utils.UrlUtil : Parse host error. url: hnb.1000328.view-ui.nwise.hnb...{02-26","content":"2024-02-26 14:46:26.588 ERROR 3205 --- [ConsumeMessageThread_16] com.xxx.jy.utils.UrlUtil : Parse host error. url: hnb.1000328.x-ui.x.x...{02-26","source_type":"file"} Exception in thread "main" jakarta.ws.rs.ProcessingException: Error reading entity from input stream. at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:934) at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:879) at org.glassfish.jersey.client.ClientResponse.readEntity(ClientResponse.java:340) at org.glassfish.jersey.client.InboundJaxrsResponse$2.call(InboundJaxrsResponse.java:104) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:205) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:365) at org.glassfish.jersey.client.InboundJaxrsResponse.runInScopeIfPossible(InboundJaxrsResponse.java:244) at org.glassfish.jersey.client.InboundJaxrsResponse.readEntity(InboundJaxrsResponse.java:101) at com.manticoresearch.client.ApiClient.deserialize(ApiClient.java:1048) at com.manticoresearch.client.ApiClient.invokeAPI(ApiClient.java:1233) at com.manticoresearch.client.api.IndexApi.insertWithHttpInfo(IndexApi.java:196) at com.manticoresearch.client.api.IndexApi.insert(IndexApi.java:168) at com.xxx.security.starlog.DataFetcherWorker.fetchData(DataFetcherWorker.java:68) at com.xxx.security.starlog.MainWorker.main(MainWorker.java:11) Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of typecom.manticoresearch.client.model.SuccessResponsefrom Array value (tokenJsonToken.START_ARRAY) at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 1] at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1752) at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1526) at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1473) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeFromArray(BeanDeserializer.java:656) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:211) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187) at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) at com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:2079) at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1229) at org.glassfish.jersey.jackson.internal.jackson.jaxrs.base.ProviderBase.readFrom(ProviderBase.java:831) at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.invokeReadFrom(ReaderInterceptorExecutor.java:233) at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.aroundReadFrom(ReaderInterceptorExecutor.java:212) at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:132) at org.glassfish.jersey.message.internal.MessageBodyFactory.readFrom(MessageBodyFactory.java:1072) at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:919) ... 15 more

how can i fix it ~~

FriskKiddo commented 8 months ago

Hm, the manticoresearch-java version i used is 5.0.0

FriskKiddo commented 8 months ago

manticore version is 6.2.13 b826cafe8@231007 dev (columnar 2.2.5 b8be4eb@230928) (secondary 2.2.5 b8be4eb@230928)

Nick-S-2018 commented 8 months ago

It appears that you get an error response from Manticore instead of expected success response, but the client cannot correctly process such situation now. We are aware of this problem and already working on it in the related issue https://github.com/manticoresoftware/manticoresearch/issues/1806 . As soon as we finish it we'll publish appropriate updates for all our clients.

You can try to check if this is really the case by sending a similar HTTP JSON request to your Manticore server (e.g. using curl) and seeing the response it returns.

FriskKiddo commented 8 months ago

Hm,i have tried to use curl to send a request to my Manticore server, eg curl -X POST http://xxxxx.com/insert \ -H "Content-Type: application/json" \ -H "Accept: application/json" \ -d '{ "index": "csis_log", "doc": { // ignored } }' I got a success response as follow: {"_index":"csis_log","_id":6703911121000730511,"created":true,"result":"created","status":201} However, i got a same exception by Java client. Finally, thank you very much! I will appreciate it if you could publish updates as soon as possible.

Nick-S-2018 commented 8 months ago

It seems the requests in your application are a bit different, comparing with the example above (e.g., there's the cluster field used) which may be the reason of the different outcome. Anyway, I think, this will beсome clear when we complete the updates I mentioned.

sanikolaev commented 8 months ago

Blocked by https://github.com/manticoresoftware/manticoresearch/issues/1806

sanikolaev commented 6 months ago

Blocked by https://github.com/manticoresoftware/manticoresearch/issues/1806

That's done.

Nick-S-2018 commented 6 months ago

@FriskKiddo We've implemented the updates I mentioned above and altered the structure of responses Manticore sends to the client which should solve your issue. You can try it with Manticore dev version now.

Nick-S-2018 commented 5 months ago

Fixed in https://github.com/manticoresoftware/manticoresearch/pull/1869