search

elastic / elasticsearch-java

Official Elasticsearch Java Client
Apache License 2.0
397 stars 228 forks source link

I can't disable the 'type#name' format in the deserialization of the response #827

Closed powfyy closed 1 month ago

powfyy commented 1 month ago

Java API client version

8.13.2

Java version

17

Elasticsearch Version

8.12.2

Problem description

Hello. I want to disable the 'type#name' format when deserializing the response. To do this, I redefined the creation of jsonpMapper and added an attribute 'JsonpMapperFeatures.SERIALIZE_TYPED_KEYS = false', but the error did not disappear. I see https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/serialize-without-typed-keys.html

ElasticConfig:

@RequiredArgsConstructor
@EnableConfigurationProperties({ConnectProperties.class, IndexProperties.class})
public class ElasticsearchConfig extends ElasticsearchConfiguration {

  private final ConnectProperties properties;

  @Override
  public ClientConfiguration clientConfiguration() {
    return ClientConfiguration.builder()
      .connectedTo(properties.getHost() + ":" + properties.getPort())
      .withBasicAuth(properties.getUser(), properties.getPassword())
      .withConnectTimeout(properties.getConnectTimeout())
      .withSocketTimeout(properties.getSocketTimeout())
      .build();
  }

  @Override
  public JsonpMapper jsonpMapper() {
    ObjectMapper objectMapper = (new ObjectMapper()).configure(SerializationFeature.INDENT_OUTPUT, false).setSerializationInclusion(JsonInclude.Include.ALWAYS);
    JsonpMapper jsonpMapper = new JacksonJsonpMapper(objectMapper);
    return jsonpMapper.withAttribute(JsonpMapperFeatures.SERIALIZE_TYPED_KEYS, false);
  }
}

Class with request

@Slf4j
@Service
@RequiredArgsConstructor
public class ElasticsearchSuggestService {

  private final ElasticsearchClient elasticsearchClient;
  @Getter
  private final SuggestProperties suggestProperties;

  private final TransportOptions transportOptions = new DefaultTransportOptions.Builder()
    .setParameter("filter_path", "suggest.*.options.text")
    .setParameter("typed_keys", "false")
    .build();

  public void executeRequestAndFillContext(SearchTemplateRequest searchTemplateRequest, SuggestPipelineContext context) {
    try {
      var response = elasticsearchClient
        .withTransportOptions(transportOptions).searchTemplate(searchTemplateRequest, Void.class);
    } catch (Exception ex) {
      log.error("Error during request", ex);
      context.setFinished(true);
    }
  }
}

Logs:

co.elastic.clients.transport.TransportException: node: http://***.***.**.***:80/, status: 200, [es/search_template] Failed to decode response
    at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:404) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.transport.ElasticsearchTransportBase.getApiResponse(ElasticsearchTransportBase.java:363) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.transport.ElasticsearchTransportBase.performRequest(ElasticsearchTransportBase.java:147) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.elasticsearch.ElasticsearchClient.searchTemplate(ElasticsearchClient.java:2080) ~[elasticsearch-java-8.13.2.jar:na]
    at com.vkusomania.search.service.service.suggest.ElasticsearchSuggestService.executeRequestAndFillContext(ElasticsearchSuggestService.java:33) ~[main/:na]
    at com.vkusomania.search.service.service.suggest.steps.AutocompleteStep.process(AutocompleteStep.java:24) ~[main/:na]
    at com.vkusomania.search.service.service.suggest.steps.AbstractSuggestStep.execute(AbstractSuggestStep.java:18) ~[main/:na]
    at com.vkusomania.search.service.service.suggest.SuggestService.lambda$getSuggests$1(SuggestService.java:64) ~[main/:na]
    at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na]
    at com.vkusomania.search.service.service.suggest.SuggestService.getSuggests(SuggestService.java:64) ~[main/:na]
    at com.vkusomania.search.service.controller.Controller.getSuggests(Controller.java:45) ~[main/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:255) ~[spring-web-6.1.6.jar:6.1.6]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:188) ~[spring-web-6.1.6.jar:6.1.6]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.6.jar:6.1.6]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:926) ~[spring-webmvc-6.1.6.jar:6.1.6]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:831) ~[spring-webmvc-6.1.6.jar:6.1.6]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.1.6.jar:6.1.6]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089) ~[spring-webmvc-6.1.6.jar:6.1.6]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979) ~[spring-webmvc-6.1.6.jar:6.1.6]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-6.1.6.jar:6.1.6]
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) ~[spring-webmvc-6.1.6.jar:6.1.6]
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564) ~[tomcat-embed-core-10.1.20.jar:6.0]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.1.6.jar:6.1.6]
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-10.1.20.jar:6.0]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:150) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-embed-websocket-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:175) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:150) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.1.6.jar:6.1.6]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.6.jar:6.1.6]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:175) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:150) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.1.6.jar:6.1.6]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.6.jar:6.1.6]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:175) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:150) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109) ~[spring-web-6.1.6.jar:6.1.6]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.6.jar:6.1.6]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:175) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:150) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-6.1.6.jar:6.1.6]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.6.jar:6.1.6]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:175) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:150) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1736) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-embed-core-10.1.20.jar:10.1.20]
    at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.core.SearchTemplateResponse: Property name 'CATEGORY' is not in the 'type#name' format. Make sure the request has 'typed_keys' set. (JSON path: suggest.CATEGORY.CATEGORY) (line no=1, column no=25, offset=-1)
    at co.elastic.clients.json.ExternallyTaggedUnion.lambda$arrayMapDeserializer$0(ExternallyTaggedUnion.java:151) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.json.JsonpDeserializer$3.deserialize(JsonpDeserializer.java:136) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:192) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:148) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.transport.endpoints.EndpointWithResponseMapperAttr$1.deserialize(EndpointWithResponseMapperAttr.java:56) ~[elasticsearch-java-8.13.2.jar:na]
    at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:399) ~[elasticsearch-java-8.13.2.jar:na]
    ... 64 common frames omitted
l-trotta commented 1 month ago

Hello! Quoting from this similar issue:

unfortunately the java client cannot deserialize aggregation results without the typed_keys option

As the documentation shows, disabling typed_keys can be useful when the original response in json/string format is needed, but still, the java client cannot deserialize it.