search

redis / redis-om-spring

Spring Data Redis extensions for better search, documents models, and more
MIT License
609 stars 94 forks source link

Added a clauseTemplate for Numeric fieldType to skip the escaping of … #514

Closed foogaro closed 1 month ago

foogaro commented 1 month ago

…the characters.

Using Redis OM Spring 0.9.4, I've implemented the following repository:

@Repository
public interface BeersRepository extends CrudRepository<Beer, String> {

    List<Beer> searchBeerByName(String name);
    Iterable<Beer> findByAbv(String abv);
    Iterable<Beer> findByAbvGreaterThanEqual(String abvGTE);
    Iterable<Beer> findByAbvLessThanEqual(String abvLTE);
    Iterable<Beer> findByAbvBetween(String abvGT, String abvLT);
    Iterable<Beer> findByIbu(String ibu);
    Iterable<Beer> findByIbuGreaterThanEqual(String ibuGTE);
    Iterable<Beer> findByIbuLessThanEqual(String ibuLTE);
    Iterable<Beer> findByIbuBetween(String ibuGT, String ibuLT);

}

for the following entity:

@RedisHash("beer")
public class Beer {

    @Id
    private String id;
    @NumericIndexed(sortable = true)
    private String abv; //Alcohol by volume (ABV).
    private Integer available_id;
    private String beer_variation_id;
    private String create_date;
    private String description;
    private String food_parings;
    private Integer glassware_id;
    @NumericIndexed(sortable = true)
    private String ibu; //Bitterness
    private String is_organic;
    private String is_retired;
    @Searchable
    private String name;
    private String name_display;
    private String original_gravity;
    private String serving_temperature;
    private String serving_temperature_display;
    private Integer srm_id;
    private String status;
    private String status_display;
    private Integer style_id;
    private String update_date;
    private Integer year;

}

The search is made available through a REST API as follows:

    @GetMapping("/by-abv/{abvGT}/{abvLT}")
    public ResponseEntity findByAbvBetween(@PathVariable("abvGT") String abvGT, @PathVariable("abvLT") String abvLT) {
        Iterable<Beer> beers = service.findByAbvBetween(abvGT, abvLT);
        if (beers != null) {
            return ResponseEntity.ok(beers);
        } else {
            return ResponseEntity.notFound().build();
        }
    }

Issue

Using the curl command I receive the following error:

curl -X GET http://localhost:8080/beers/by-abv/21.9/22.1
{"timestamp":"2024-10-07T13:52:37.806+00:00","status":500,"error":"Internal Server Error","path":"/beers/by-abv/21.9/22.1"}%

Using the Redis Insight Profiler I see the following command being invoked on Redis:

15:52:37.788 [0 192.168.65.1:26960] "FT.SEARCH" "com.foogaro.panel.model.BeerIdx" "@abv:[21\\.9 22\\.1]" "LIMIT" "0" "10000" "DIALECT" "1"

And at application level I get the following error in the console:

2024-10-07T15:52:37.793+02:00 ERROR 62679 --- [beers] [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.dao.InvalidDataAccessApiUsageException: Syntax error at offset 6 near 21\.9] with root cause

redis.clients.jedis.exceptions.JedisDataException: Syntax error at offset 6 near 21\.9
    at redis.clients.jedis.Protocol.processError(Protocol.java:105) ~[jedis-5.1.0.jar:na]
    at redis.clients.jedis.Protocol.process(Protocol.java:162) ~[jedis-5.1.0.jar:na]
    at redis.clients.jedis.Protocol.read(Protocol.java:221) ~[jedis-5.1.0.jar:na]
    at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:350) ~[jedis-5.1.0.jar:na]
    at redis.clients.jedis.Connection.getOne(Connection.java:332) ~[jedis-5.1.0.jar:na]
    at redis.clients.jedis.Connection.executeCommand(Connection.java:137) ~[jedis-5.1.0.jar:na]
    at redis.clients.jedis.executors.DefaultCommandExecutor.executeCommand(DefaultCommandExecutor.java:24) ~[jedis-5.1.0.jar:na]
    at redis.clients.jedis.UnifiedJedis.executeCommand(UnifiedJedis.java:250) ~[jedis-5.1.0.jar:na]
    at redis.clients.jedis.UnifiedJedis.ftSearch(UnifiedJedis.java:3719) ~[jedis-5.1.0.jar:na]
    at com.redis.om.spring.ops.search.SearchOperationsImpl.search(SearchOperationsImpl.java:46) ~[redis-om-spring-0.9.4.jar:na]
    at com.redis.om.spring.repository.query.RedisEnhancedQuery.executeQuery(RedisEnhancedQuery.java:489) ~[redis-om-spring-0.9.4.jar:na]
    at com.redis.om.spring.repository.query.RedisEnhancedQuery.execute(RedisEnhancedQuery.java:387) ~[redis-om-spring-0.9.4.jar:na]
    at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:170) ~[spring-data-commons-3.3.2.jar:3.3.2]
    at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:158) ~[spring-data-commons-3.3.2.jar:3.3.2]
    at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:164) ~[spring-data-commons-3.3.2.jar:3.3.2]
    at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:143) ~[spring-data-commons-3.3.2.jar:3.3.2]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[spring-aop-6.1.11.jar:6.1.11]
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) ~[spring-aop-6.1.11.jar:6.1.11]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[spring-aop-6.1.11.jar:6.1.11]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:223) ~[spring-aop-6.1.11.jar:6.1.11]
    at jdk.proxy2/jdk.proxy2.$Proxy86.findByAbvBetween(Unknown Source) ~[na:na]
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:354) ~[spring-aop-6.1.11.jar:6.1.11]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196) ~[spring-aop-6.1.11.jar:6.1.11]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-6.1.11.jar:6.1.11]
    at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:138) ~[spring-tx-6.1.11.jar:6.1.11]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[spring-aop-6.1.11.jar:6.1.11]
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) ~[spring-aop-6.1.11.jar:6.1.11]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[spring-aop-6.1.11.jar:6.1.11]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:223) ~[spring-aop-6.1.11.jar:6.1.11]
    at jdk.proxy2/jdk.proxy2.$Proxy86.findByAbvBetween(Unknown Source) ~[na:na]
    at com.foogaro.panel.service.BeersService.findByAbvBetween(BeersService.java:42) ~[classes/:na]
    at com.foogaro.panel.controller.BeersController.findByAbvBetween(BeersController.java:54) ~[classes/:na]
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:255) ~[spring-web-6.1.11.jar:6.1.11]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:188) ~[spring-web-6.1.11.jar:6.1.11]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.11.jar:6.1.11]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:926) ~[spring-webmvc-6.1.11.jar:6.1.11]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:831) ~[spring-webmvc-6.1.11.jar:6.1.11]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.1.11.jar:6.1.11]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089) ~[spring-webmvc-6.1.11.jar:6.1.11]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979) ~[spring-webmvc-6.1.11.jar:6.1.11]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-6.1.11.jar:6.1.11]
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) ~[spring-webmvc-6.1.11.jar:6.1.11]
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.1.11.jar:6.1.11]
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-embed-websocket-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.1.11.jar:6.1.11]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.11.jar:6.1.11]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.1.11.jar:6.1.11]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.11.jar:6.1.11]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-6.1.11.jar:6.1.11]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.11.jar:6.1.11]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:389) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:904) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-embed-core-10.1.26.jar:10.1.26]
    at java.base/java.lang.Thread.run(Thread.java:1583) ~[na:na]

Expected

Expected behaviour is to generate proper Redis Search query without escaping the decimal dot separator, as follows:

FT.SEARCH com.foogaro.panel.model.BeerIdx "@abv:[21.9 22.1]" LIMIT 0 10000 DIALECT 1

Fix

In the com.redis.om.spring.repository.query.clause.QueryClause enum, in the public String prepareQuery(String field, Object... params) method, add a condition for FieldType.NUMERIC to do not execute the QueryUtils.escape(ObjectUtils.asString(param, converter)) check on the param.

Result

Query executed properly and result given.

Redis Insight Profiler:

15:56:00.969 [0 192.168.65.1:55214] "FT.SEARCH" "com.foogaro.panel.model.BeerIdx" "@abv:[21.9 22.1]" "LIMIT" "0" "10000" "DIALECT" "1"

curl:

curl -X GET http://localhost:8080/beers/by-abv/21.9/22.1
[{"id":"wSkM9L","abv":"22","available_id":1,"beer_variation_id":null,"create_date":"2012-10-21 20:34:15","description":"Dark Strong Ale","food_parings":null,"glassware_id":4,"ibu":"48","is_organic":"N","is_retired":"N","name":"OMG","name_display":"OMG","original_gravity":"1.185","serving_temperature":"cold","serving_temperature_display":"Cold - (4-7C/39-45F)","srm_id":16,"status":"verified","status_display":"Verified","style_id":34,"update_date":"2019-06-28 15:13:30","year":null}]%