rework of BLOB support

[hohwille]

From #52 we now have BLOB support in our sample application. However, there is some rework to do:

1. BLOBs have to be streamed properly:

// REVIEW arturk88 (hohwille) we need to find another way to stream the blob without loading into heap. // https://github.com/oasp/oasp4j-sample/pull/45 byte[] data = IOUtils.readBytesFromStream(blob.getBinaryStream());

1. We need to revisit our authorization:

   @Path("/product/{id}/picture") // REVIEW arturk88 (hohwille) wrong permission, we need to create SAVE_PRODUCT_PICTURE and DELETE_PRODUCT_PICTURE @RolesAllowed(PermissionConstants.SAVE_PRODUCT) public void deleteProductPicture(long productId) {

2. We need to centralize BLOB handling code. I am taking about the transformation from Blob to MultipartBody (see 1.) that should go to a central place. E.g. some AbstractRestServiceImpl (IMHO better ease of use compare to a spring-bean that has to be injected before using).

[julianmetzler]

// REVIEW arturk88 (hohwille) we need to find another way to stream the blob without loading into heap.

Regarding this problem I didn't find a solution.

[maihacke]

Could you please explain, whats open here?

[julianmetzler]

Currently when a file is uploaded or downloaded as a BLOB it is loaded completely to memory and then it is written to the database.

Blob blob = new SerialBlob(IOUtils.readBytesFromStream(picture));

To support files that are larger than memory it would be better to find a solution to write a BLOB to the database without loading it to memory completely.

[hohwille]

To further explain what @julianmetzler said: We already tried to stream the data instead of loading into byte[] but a naive approach is simply not working for technical reasons (JAX-RS and JPA do not go well together here - maybe TX is closed in the middle of async processing or whatever, download receives truncated garbage if we use real streaming instead of byte[]).

[maybeec]

download receives truncated garbage if we use real streaming instead of byte[]

but saving (streaming from REST to database) works?

[maybeec]

The following is an interesting discussion about the JDBC specification according to transactions and the validity of blobs outside of transactions: http://stackoverflow.com/questions/30356840/reading-from-a-jdbc-blob-after-leaving-the-spring-transaction

Given this nice summary of experiences and facts, which matches my experiences until now, there is no universal valid way of streaming blobs from database directly over a RESTful service to the client based on JDBC and JAX RS without reading the blob's contents to memory.

[MarcoRose]

We consolidate the simple non-streaming solution for now, document it and implement a sample with the OASP4js-restaurant. This issue can then be closed and reopened for a streaming approach.

[sesslinger]

I documented BLOB support in OASP using the example of our restaurant application by adding a new guide to the wiki.

[hohwille]

I still see an open issue with the byte[] approach. This is not a reasonable solution. Closing this issue is just hiding a pain-point we have not yet resolved. For tidy up we can file a new issue as everything else around this is already complete.

[hohwille]

As we even have dropped it in MTS and went for an anti-pattern to store BLOBs as base64 string I am going to reopen this issue.

[hohwille]

Given this nice summary of experiences and facts, which matches my experiences until now, there is no universal valid way of streaming blobs from database directly over a RESTful service to the client based on JDBC and JAX RS without reading the blob's contents to memory.

If direct streaming does not work with Java frameworks (what actually sucks) then we should use some library to workaround it. Ages ago I used commons-fileupload what worked fine (it kept the uploaded BLOB in memory until some configurable size was reached and from that point on it streamed the file into a temporary file from where it was processed further). IMHO we do have a reasonable issue here. Many CSD projects have already solved this properly in the past. So why can't we provide a general solution? Probably we can just harvest an existing project solution. Also I remember there were some talks about this on Yammer where this needed to be solved but it never reached devon.

[maihacke]

I created an example which demonstrates working blob streaming with oracle and apache cxf: https://github.com/maihacke/devon4j-blob-streaming/tree/develop