RestBuilder default constructor breaks JSON converters/marshallers since 6.0.5

benorama commented 7 years ago

Steps to Reproduce

Add compile 'org.grails:grails-datastore-rest-client' dependency
Define a restBuilder(RestBuilder) bean in your spring resources.groovy
Try to use JSON marshalling (ex.: as JSON)

Expected Behaviour

It should render JSON.

Actual Behaviour

It throws a StringIndexOutOfBoundsException exception.

Environment Information

Operating System: Mac OSX 10.12.3
Grails Version: 3.2.5
JDK Version: 1.8.0_25

Example Application

https://github.com/benorama/grails-restbuilder-json-issue

benorama commented 7 years ago

This is probably linked to #825 and #536

graemerocher commented 7 years ago

Can probably be worked around by simply using new RestBuilder() instead of a bean, but yes I agree not great

benorama commented 7 years ago

We were using new RestBuilder() and it was also messing up our JSON marshallers.

Our workaround is to duplicate the RestBuilder class in our own package and removing from the constructor:

def currentConfiguration = ConvertersConfigurationHolder.getConverterConfiguration(JSON)
if(currentConfiguration instanceof DefaultConverterConfiguration) {
    // init manually
    DefaultConverterConfiguration defaultConfig = ((DefaultConverterConfiguration)currentConfiguration)
    defaultConfig.registerObjectMarshaller(new MapMarshaller())
    defaultConfig.registerObjectMarshaller(new ArrayMarshaller());
    defaultConfig.registerObjectMarshaller(new ByteArrayMarshaller());
    defaultConfig.registerObjectMarshaller(new CollectionMarshaller());
    defaultConfig.registerObjectMarshaller(new GroovyBeanMarshaller());
}

It solves the issue on our apps.

benorama commented 7 years ago

If it can help, when turning on grails.converters.default.circular.reference.behaviour = "EXCEPTION" I've got this exception when using JSON marshaller/converters on a simple Map:

org.grails.web.converters.exceptions.ConverterException: Circular Reference detected: class java.util.LinkedHashMap
        at grails.converters.JSON.handleCircularRelationship(JSON.java:354)
        at grails.converters.JSON.value(JSON.java:176)
        at grails.converters.JSON.convertAnother(JSON.java:144)
        at org.grails.web.converters.marshaller.json.GroovyBeanMarshaller.marshalObject(GroovyBeanMarshaller.java:67)
        at org.grails.web.converters.marshaller.json.GroovyBeanMarshaller.marshalObject(GroovyBeanMarshaller.java:39)
        at grails.converters.JSON.value(JSON.java:184)
        at grails.converters.JSON.convertAnother(JSON.java:144)
        at org.grails.web.converters.marshaller.json.GroovyBeanMarshaller.marshalObject(GroovyBeanMarshaller.java:67)
        at org.grails.web.converters.marshaller.json.GroovyBeanMarshaller.marshalObject(GroovyBeanMarshaller.java:39)
        at grails.converters.JSON.value(JSON.java:184)
        at grails.converters.JSON.convertAnother(JSON.java:144)
        at org.grails.web.converters.marshaller.json.MapMarshaller.marshalObject(MapMarshaller.java:45)
        at org.grails.web.converters.marshaller.json.MapMarshaller.marshalObject(MapMarshaller.java:30)
        at grails.converters.JSON.value(JSON.java:184)
        at grails.converters.JSON.render(JSON.java:119)
        at org.grails.web.converters.AbstractConverter.toString(AbstractConverter.java:109)

jameskleeh commented 7 years ago

The issue is caused because the GroovyBeanMarshaller is failing to marshal a org.grails.config.NavigableMap$NullSafeNavigator. The marshaller that is intended for maps should be used instead. I think it's an ordering issue

jameskleeh commented 7 years ago

Fixed by https://github.com/grails/grails-data-mapping/commit/0be3beae85c87f9a03f615c50553852f438fcfd3

sarbogast commented 7 years ago

@jameskleeh FYI, Jeff's solution meant changing a lot of my tests to make them work with the API of this other RxRestBuilder thingy. So instead I went with @benorama's workaround and injected the solution in https://github.com/grails/grails-data-mapping/commit/0be3beae85c87f9a03f615c50553852f438fcfd3, and I got 15 out of 18 tests to pass again. The other 3 fail with another JSON related exception:

java.lang.RuntimeException: org.grails.web.converters.exceptions.ConverterException: Error converting Bean with class org.springframework.beans.GenericTypeAwarePropertyDescriptor
    at org.grails.web.converters.AbstractConverter.toString(AbstractConverter.java:111)
    at org.codehaus.groovy.runtime.InvokerHelper.format(InvokerHelper.java:628)
    at org.codehaus.groovy.runtime.InvokerHelper.format(InvokerHelper.java:575)
    at org.codehaus.groovy.runtime.InvokerHelper.toString(InvokerHelper.java:130)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.toString(DefaultGroovyMethods.java:13696)
    at grails.artefact.controller.support.ResponseRenderer$Trait$Helper.render(ResponseRenderer.groovy:256)
    at grails.artefact.controller.support.ResponseRenderer$Trait$Helper$render$1.call(Unknown Source)

I'll continue to investigate.

jameskleeh commented 7 years ago

@sarbogast Feel free to open a new issue with a sample project in grails-data-mapping

sarbogast commented 7 years ago

The root cause seems to be way deeper than that and it's harder to identify what's causing it. But I'll try:

java.lang.IllegalAccessException: Class org.grails.web.converters.marshaller.json.GenericJavaBeanMarshaller can not access a member of class org.springframework.beans.GenericTypeAwarePropertyDescriptor with modifiers "public"
    at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)
    at java.lang.reflect.AccessibleObject.slowCheckMemberAccess(AccessibleObject.java:296)
    at java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:288)
    at java.lang.reflect.Method.invoke(Method.java:490)
    at org.grails.web.converters.marshaller.json.GenericJavaBeanMarshaller.marshalObject(GenericJavaBeanMarshaller.java:48)
    at org.grails.web.converters.marshaller.json.GenericJavaBeanMarshaller.marshalObject(GenericJavaBeanMarshaller.java:34)
    at grails.converters.JSON.value(JSON.java:184)
    at grails.converters.JSON.convertAnother(JSON.java:144)
    at org.grails.web.converters.marshaller.json.ArrayMarshaller.marshalObject(ArrayMarshaller.java:41)
    at org.grails.web.converters.marshaller.json.ArrayMarshaller.marshalObject(ArrayMarshaller.java:30)
    at grails.converters.JSON.value(JSON.java:184)
    at grails.converters.JSON.convertAnother(JSON.java:144)
    at org.grails.web.converters.marshaller.json.GenericJavaBeanMarshaller.marshalObject(GenericJavaBeanMarshaller.java:50)
    at org.grails.web.converters.marshaller.json.GenericJavaBeanMarshaller.marshalObject(GenericJavaBeanMarshaller.java:34)
    at grails.converters.JSON.value(JSON.java:184)
    at grails.converters.JSON.convertAnother(JSON.java:144)
    at org.grails.web.converters.marshaller.json.GroovyBeanMarshaller.marshalObject(GroovyBeanMarshaller.java:67)
    at org.grails.web.converters.marshaller.json.GroovyBeanMarshaller.marshalObject(GroovyBeanMarshaller.java:39)
    at grails.converters.JSON.value(JSON.java:184)
    at grails.converters.JSON.convertAnother(JSON.java:144)
    at org.grails.web.converters.marshaller.json.GroovyBeanMarshaller.marshalObject(GroovyBeanMarshaller.java:67)
    at org.grails.web.converters.marshaller.json.GroovyBeanMarshaller.marshalObject(GroovyBeanMarshaller.java:39)
    at grails.converters.JSON.value(JSON.java:184)
    at grails.converters.JSON.convertAnother(JSON.java:144)
    at org.grails.web.converters.marshaller.json.CollectionMarshaller.marshalObject(CollectionMarshaller.java:41)
    at org.grails.web.converters.marshaller.json.CollectionMarshaller.marshalObject(CollectionMarshaller.java:30)
    at grails.converters.JSON.value(JSON.java:184)
    at grails.converters.JSON.convertAnother(JSON.java:144)
    at org.grails.web.converters.marshaller.json.MapMarshaller.marshalObject(MapMarshaller.java:45)
    at org.grails.web.converters.marshaller.json.MapMarshaller.marshalObject(MapMarshaller.java:30)
    at grails.converters.JSON.value(JSON.java:184)
    at grails.converters.JSON.convertAnother(JSON.java:144)
    at org.grails.web.converters.marshaller.json.CollectionMarshaller.marshalObject(CollectionMarshaller.java:41)
    at org.grails.web.converters.marshaller.json.CollectionMarshaller.marshalObject(CollectionMarshaller.java:30)
    at grails.converters.JSON.value(JSON.java:184)
    at grails.converters.JSON.convertAnother(JSON.java:144)
    at org.grails.web.converters.marshaller.json.MapMarshaller.marshalObject(MapMarshaller.java:45)
    at org.grails.web.converters.marshaller.json.MapMarshaller.marshalObject(MapMarshaller.java:30)
    at grails.converters.JSON.value(JSON.java:184)
    at grails.converters.JSON.render(JSON.java:119)
    at org.grails.web.converters.AbstractConverter.toString(AbstractConverter.java:109)
    at org.codehaus.groovy.runtime.InvokerHelper.format(InvokerHelper.java:628)
    at org.codehaus.groovy.runtime.InvokerHelper.format(InvokerHelper.java:575)
    at org.codehaus.groovy.runtime.InvokerHelper.toString(InvokerHelper.java:130)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.toString(DefaultGroovyMethods.java:13696)
    at grails.artefact.controller.support.ResponseRenderer$Trait$Helper.render(ResponseRenderer.groovy:256)
    at grails.artefact.controller.support.ResponseRenderer$Trait$Helper$render$1.call(Unknown Source)
    at com.adessa.unbox.api.employee.EmployeeImportController.render(EmployeeImportController.groovy)
    at grails.artefact.controller.support.ResponseRenderer$render.callCurrent(Unknown Source)
    at com.adessa.unbox.api.employee.EmployeeImportController.importEmployees(EmployeeImportController.groovy:37)
    at com.adessa.unbox.api.employee.EmployeeImportController.importEmployees(EmployeeImportController.groovy)
    at org.grails.core.DefaultGrailsControllerClass$MethodHandleInvoker.invoke(DefaultGrailsControllerClass.java:222)
    at org.grails.core.DefaultGrailsControllerClass.invoke(DefaultGrailsControllerClass.java:187)
    at org.grails.web.mapping.mvc.UrlMappingsInfoHandlerAdapter.handle(UrlMappingsInfoHandlerAdapter.groovy:90)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
    at org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:651)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at javax.servlet.FilterChain$doFilter.call(Unknown Source)
    at grails.plugin.springsecurity.rest.RestLogoutFilter.doFilter(RestLogoutFilter.groovy:80)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:105)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at javax.servlet.FilterChain$doFilter.call(Unknown Source)
    at grails.plugin.springsecurity.rest.RestTokenValidationFilter.processFilterChain(RestTokenValidationFilter.groovy:118)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190)
    at grails.plugin.springsecurity.rest.RestTokenValidationFilter.doFilter(RestTokenValidationFilter.groovy:84)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:169)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at javax.servlet.FilterChain$doFilter.call(Unknown Source)
    at grails.plugin.springsecurity.rest.RestAuthenticationFilter.doFilter(RestAuthenticationFilter.groovy:143)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at grails.plugin.springsecurity.web.authentication.logout.MutableLogoutFilter.doFilter(MutableLogoutFilter.groovy:62)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at grails.plugin.springsecurity.web.SecurityRequestHolderFilter.doFilter(SecurityRequestHolderFilter.groovy:58)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)

sarbogast commented 7 years ago

OK, I really couldn't reproduce in a sample app. But I can confirm that your fix with lower priority marshallers does not fix this exception. I found the same exception report in old versions of Grails: http://grails.1312388.n4.nabble.com/Marshallers-are-blowing-up-in-2-3-5-anyone-else-td4653954.html

Once I comment out this whole section in RestBuilder's constructor, my tests pass again.

jameskleeh commented 7 years ago

@sarbogast Without something to debug, I can't fix the issue. In the meantime I'll be adding an option to turn off the creation of the converters

sarbogast commented 7 years ago

I know, but for the life of me I couldn't isolate what is going on and why it's failing. Apparently in my case it fails to call getClass() on an object of a class I don't even know of. And impossible to figure out what was common to those 3 failing tests compared to the 117 others.

jameskleeh commented 7 years ago

@benorama @sarbogast In the next version of GORM, you can do new RestBuilder(registerConverters: false) which will prevent the converters from being intialized

benorama commented 7 years ago

That's great! Would it be possible to have a global config parameter, since you might not control instances created from plugins?

abrahaj commented 5 years ago

@benorama @sarbogast In the next version of GORM, you can do new RestBuilder(registerConverters: false) which will prevent the converters from being intialized

Run into the same issue. Solved by applying the registerConverters:false (Grails 3.3.9)

grails / grails-data-mapping