spring-projects / spring-data-jpa

Simplifies the development of creating a JPA-based data access layer.
https://spring.io/projects/spring-data-jpa/
Apache License 2.0
3.01k stars 1.42k forks source link

Identifier lookup fails for JPA proxies [DATAJPA-630] #1012

Closed spring-projects-issues closed 9 years ago

spring-projects-issues commented 10 years ago

Petar Tahchiev opened DATAJPA-630 and commented

Hi guys,

I'm testing with the latest 2.1.0.BUILD-SNAPSHOT and when I try to open any of these links:

http://localhost:8111/rest/product/563845292892480/catalogVersion http://localhost:8111/rest/product/563845292892480/contentUnit http://localhost:8111/rest/product/563845292892480/variantType

I get this json:

{
    "cause": null,
    "message": "Id must be assignable to Serializable! Object of class [null] must be an instance of interface java.io.Serializable"
}

and this exception in the log:

[ERROR] Id must be assignable to Serializable! Object of class [null] must be an instance of interface java.io.Serializable
java.lang.IllegalArgumentException: Id must be assignable to Serializable! Object of class [null] must be an instance of interface java.io.Serializable
    at org.springframework.util.Assert.isInstanceOf(Assert.java:339)
    at org.springframework.data.rest.webmvc.support.RepositoryEntityLinks.linkToSingleResource(RepositoryEntityLinks.java:147)
    at org.springframework.data.rest.webmvc.PersistentEntityResourceAssembler.getSelfLinkFor(PersistentEntityResourceAssembler.java:88)
    at org.springframework.data.rest.webmvc.PersistentEntityResourceAssembler.toResource(PersistentEntityResourceAssembler.java:64)
    at org.springframework.data.rest.webmvc.RepositoryPropertyReferenceController$1.apply(RepositoryPropertyReferenceController.java:141)
    at org.springframework.data.rest.webmvc.RepositoryPropertyReferenceController$1.apply(RepositoryPropertyReferenceController.java:110)
    at org.springframework.data.rest.webmvc.RepositoryPropertyReferenceController.doWithReferencedProperty(RepositoryPropertyReferenceController.java:463)
    at org.springframework.data.rest.webmvc.RepositoryPropertyReferenceController.followPropertyReference(RepositoryPropertyReferenceController.java:148)
    at sun.reflect.GeneratedMethodAccessor199.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:690)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:945)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:876)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:852)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:717)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1644)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:154)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:110)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:50)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:108)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:344)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:261)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1632)
    at com.xxxxx.xxxx.xxxxx.xxxx.xxxxxx.filter.CorsFilter.doFilterInternal(CorsFilter.java:34)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:108)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1632)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:108)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1624)
    at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:164)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1615)
    at org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1615)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:568)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1110)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:479)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1044)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:199)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:459)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:281)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
    at org.eclipse.jetty.io.AbstractConnection$1.run(AbstractConnection.java:505)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
    at java.lang.Thread.run(Thread.java:722)

Issue Links:

3 votes, 11 watchers

spring-projects-issues commented 10 years ago

Dave Hallam commented

Created related (possibly duplicate) issue DATAGRAPH-461 on the SDN project

spring-projects-issues commented 10 years ago

Johannes Hiemer commented

Facing the same over here with:

@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "order_id")
private Order order;
@OneToMany(fetch = FetchType.LAZY, mappedBy = "order")
private List<Vapp> vapps;
{
rel: "vapps",
href: "http://localhost:8080/vw-web/orders/256/vapps"
}

Calling the vapps from order works perfectly.

{
rel: "order",
href: "http://localhost:8080/vw-web/vapps/233/order"
}

The other way round the above exception occurs.

spring-projects-issues commented 10 years ago

Petar Tahchiev commented

This happens because in PersistentEntityResourceAssembler:200 the instance is a jpa lazy-loaded proxy:

BeanWrapper<Object> wrapper = BeanWrapper.create(instance, null);
Object id = wrapper.getProperty(entity.getIdProperty());

So then in wrapper.getProperty:125 we have:

Field field = property.getField();
ReflectionUtils.makeAccessible(field);
obj = ReflectionUtils.getField(field, bean);

which returns null. I tried setting: @Basic(fetch = FetchType.EAGER) in my entity but it had no effect. I guess the solution would be to get the instance and then the pk of the instance in the same transaction

spring-projects-issues commented 10 years ago

Nayden gochev commented

Any updates on this issue? We are facing the same problem and I was wondering if it will be included in the Fowler release

spring-projects-issues commented 10 years ago

Manuel Garcia commented

I've had the same issue but the switching from FetchType.LAZY to EAGER solved it. My issue with this is the performance... I would rather have it working with LAZY.

Additional info: I'm using JPA with Hibernate to connect to PostgreSQL database

spring-projects-issues commented 10 years ago

Nayden gochev commented

This doesn't work for me. I have too many relations.. and to make the FetchType EAGER is not really an option.

It is shame that for more then half year no one have fixed this issue.. this is not a MAJOR issue this is more than BLOCKING for me and basically this makes the Data REST unusable with JPA.

spring-projects-issues commented 10 years ago

Kjetil Fjellheim commented

This is a critical issue for us as well. Eager is an issue as it causes to many objects to be retrieved. Do the developers have any workaround for this issue at the current time and do you have a plan to fix it?

spring-projects-issues commented 10 years ago

Oliver Drotbohm commented

Rest assured that we're looking into this. However the problem (and even more so finding an appropriate solution) is not trivial as we're basically facing some design decisions made by JPA persistence providers and all we can effectively do is duck taping these.

A temporary solution might be to use the Spring Data access control annotations (i.e. @AccessType(Type.PROPERTY)) for either the entire domain type or the identifier properties more specifically as this will cause the id lookup to happen through the getter of the property so that the JPA provider is able to detect the access and automatically triggers the lookup of the needed data.

I've created DATAJPA-619 to let this also work with JPA specific access annotations but it would be cool if you could give the approach using the Spring Data ones a spin to see if that helps mitigating the issue. Also, can anyone provide an as-tiny-as-possible test case that reproduces the error?

spring-projects-issues commented 9 years ago

Petar Tahchiev commented

Here it is:

https://github.com/paranoiabla/DATAREST-269

What I do to reproduce it is: 1) Clone it 2) mvn spring-boot:run 3) Use rest client to import 1 product: POST to http://localhost:8080/product/

{
    "id" : "123"
}

4) Use rest client to import 1 category: POST to http://localhost:8080/category/

{
   "id" : "c1",
   "product" : "http://localhost:8080/product/123"
}

5) Access http://localhost:8080/category/c1/product

spring-projects-issues commented 9 years ago

Oliver Drotbohm commented

This should be fixed in the latest snapshots. Make sure you use them for all Spring Data Commons, JPA and REST. Unfortunately we can't backport the fix as it requires new API to be introduced in Spring Data Commons. For everyone on the Dijkstra and Evans release trains using the access type workaround described above should be a reasonable workaround.

Petar Tahchiev - I could verify your sample project to work with the latest improvements. Be sure to add the Jackson Hibernate 4 module to the classpath to let Jackson render proxies appropriately

spring-projects-issues commented 9 years ago

Petar Tahchiev commented

As much as I don't like, I think we might have to reopen this :( The exception is no longer thrown, but I get a null content:

{
  "content" : null,
  "_links" : {
    "self" : {
      "href" : "http://localhost:8111/storefront/rest/media/140738852592373344"
    },
    "mediaFolder" : {
      "href" : "http://localhost:8111/storefront/rest/media/140738852592373344/mediaFolder"
    },
    "mediaContainer" : {
      "href" : "http://localhost:8111/storefront/rest/media/140738852592373344/mediaContainer"
    },
    "media_format" : {
      "href" : "http://localhost:8111/storefront/rest/media/140738852592373344/media_format"
    },
    "mediaWatermark" : {
      "href" : "http://localhost:8111/storefront/rest/media/140738852592373344/mediaWatermark"
    }
  }
}

If I stop with a debug in my RepositoryPropertyReferenceController and inspect the element I get the proper result in my browser. I'm pretty sure this is because of lazy-loading and because I'm using jpa's @Access(value = AccessType.FIELD). However, my application does not allow me to use the AccessType.PROPERTY :(

spring-projects-issues commented 9 years ago

Petar Tahchiev commented

The "content" I receive is null. Tried setting the @AccessType(value = AccessType.Type.PROPERTY) or @AccessType(value = AccessType.Type.FIELD) (the spring framework @AccessType but it had no effect)

spring-projects-issues commented 9 years ago

Thomas Darimont commented

Hi Petar,

would you mind creating a small sample application that reproduces this?

Cheers, Thomas

spring-projects-issues commented 9 years ago

Thomas Darimont commented

Hi Petar,

I was able to reproduce your problem with your initial example. The problem is that the nested lazy loading proxy is not unpacked correctly.

The code in org.springframework.data.rest.webmvc.RepositoryPropertyReferenceController.doWithReferencedProperty(RootResourceInformation, Serializable, String, Function<ReferencedProperty, ResourceSupport>, HttpMethod)

PersistentEntity<?, ?> persistentEntity = repoRequest.getPersistentEntity();
PersistentProperty<?> prop = persistentEntity.getPersistentProperty(propertyPath);
...
PersistentPropertyAccessor accessor = persistentEntity.getPropertyAccessor(domainObj);
Object propVal = accessor.getProperty(prop);

traverses the "product" property of the Category which returns the proxied version of the product instance. This instance cannot be serialized. We have to find a way to unwrap that proxy (probably still in JPA) or make it somehow serializable (in a way that works for other proxy mechanisms too...)

However I found a quick and nice workaround - since Fowler GA (which we released just yesterday) you can use @EntityGraph hints on findOne(..) Repository methods as well. This allows you to customize the fetch graph resolution on query basis. In this case you can define that you want to eagerly load an the Category graph along the product property-path. With @EntityGraph on a redeclared findOne(..) method on the CategoryRepository the problem goes away...

Now a GET request to: http://localhost:8080/category/c1/product

Returns the following response:

{
  "_links" : {
    "self" : {
      "href" : "http://localhost:8080/product/123"
    }
  }
}

I updated your example in my repo - have a look at the last commit: https://github.com/thomasdarimont/DATAREST-269/commit/4cf83e2398ef5a6c3b2abab6c8ff6492d3dfeef0#diff-641d7412bd0bb58faf8de8cb2a8ce883R14

Hope that helps.

Cheers, Thomas

spring-projects-issues commented 9 years ago

Petar Tahchiev commented

Hi Thomas,

thank you sooo much for your reply. It's good to see there is some workaround, however i don't think it's applicable in my case. I have more than 160 entities, and adding EntityGraph on each one of them sounds like a lot of hassle. On top of that, I give my platform to different customers, so I would have to explicitly tell them to provide the EntityGraph which I'm not sure they will follow. I know Hibernate (the JPA provider in my case) has some ways of initializing a proxy. For instance this is how I do it now:

Hibernate.initialize(entity);
if (entity instanceof HibernateProxy) {
    entity = (T) ((HibernateProxy) entity).getHibernateLazyInitializer().getImplementation();
}
return entity;

Would it be possible to create an interface (say ProxyInitializer) which users with different JPA implementations can implement and provide their custom implementation? Then I could inject my custom code in and Spring would use it to initialize the proxies

spring-projects-issues commented 9 years ago

Petar Tahchiev commented

BTW, same thing happens if I have an ElementCollection:

@ElementCollection(fetch = FetchType.LAZY)
@CollectionTable(name = "product_description_lv", indexes = {
    @Index(unique = false, columnList = "product_pk")
}, joinColumns = {
    @JoinColumn(unique = false, name = "product_pk", nullable = true, foreignKey = @ForeignKey(ConstraintMode.CONSTRAINT), updatable = true, insertable = true)
}, foreignKey = @ForeignKey(ConstraintMode.PROVIDER_DEFAULT))
@MapKeyColumn(precision = 0, unique = false, name = "locale", length = 255, scale = 0, nullable = false, updatable = true, insertable = true)
@MapKeyJoinColumn(unique = false, name = "language", nullable = false, foreignKey = @ForeignKey(ConstraintMode.CONSTRAINT), referencedColumnName = "isocode", updatable = true, insertable = true)
private Map<Locale, LocalizedLobValue> description = new HashMap<Locale, LocalizedLobValue>();

this map is returned as null, but if I stop in the RepositoryEntityController with a breakpoint, inspect the object and let it run, then the returned json is correct

spring-projects-issues commented 9 years ago

Oliver Drotbohm commented

Guys, let's not re-open tickets that have been shipped fixed with a release already. Also, this is very much about the rendering of JPA based proxies in Spring Data REST so we should probably continue with a ticket over there.

Thomas Darimont - Would you mind creating a ticket with the sample project you mention for us to have a look at? It's probably easier to get to the gist of things on a focussed example rather than a part of a much bigger application