JME Conversion of .obj Files on WBC Server

[GoogleCodeExporter]

I've developed, partially tested, and committed a new class: 
org.wholebrainproject.wbc.server.util.ObjToJbinFileConverter.

This is the class javadoc comment:
/**
 * This is a converter for converting .obj data files to JME binary format
 * (.jbin) and writing them to disk. The purpose is to provide server-side
 * conversion to (1) relieve the client of a repetitive operation, and 
 * (2) reduce the bandwidth demand for transferring files from the server to
 * the client. This server-side conversion can be also used in conjunction with
 * client-side file caching supported by SHA-1 values for determining if cached
 * files are the most current versions.
 * 
 * This class has a main method and is designed for batch operation as part
 * of a build process.
 * 
 * @author drlittle
 */

Not being a JME programmer, the converted files have not been tested in JME 
usage which is *essential* to validate this capability.

If this conversion is validated and we choose to exploit server-side .obj to 
JME binary files, there is considerable work that would need to be done. There 
are likely more, but these occur to me:
- Substantial client-side modification to accept and use .jbin files
- Maybe Datawrapper changes, especially the type field

As examples of file size reductions, here are two cases:
ACB.obj 602K --> ACB.jbin 133K
ATN.obj 491K --> ATN.jbin 110K

I was able to run the conversion on every .obj file except 
RGN_Melanopsin_UCSB.obj (22M) due to heap space problems. This can probably be 
solved.

Once again, there is no certainty on my part that the .jbin files can be 
imported as Node or Trimesh objects although I have used standard JME classes 
and methods to do the conversions.

Original issue reported on code.google.com by davlit0...@gmail.com on 3 Sep 2010 at 3:17

• Blocking: #238

[GoogleCodeExporter]

Awesome!  Related to your comment about datawrappers, please see issue 238.

Original comment by stephen....@gmail.com on 3 Sep 2010 at 7:09

[GoogleCodeExporter]

Expanding the datawrappers handling to support server-side file conversion, 
here are some thoughts.

Although there are some variations, datawrapper's for WBC server hosted .obj 
files have a URL for the file on the server, a type field which identifies the 
file as an OBJ file, and the file's SHA-1 value. The client uses the type field 
or the file extension to know what JME converter to apply -- ObjToJme in this 
case.

If the server has preconverted, the file URL can be changed to a .jbin 
extension and the type field can be set to JBIN, BINARY, or whatever to 
indicate it's a converted JME binary file. On seeing this this the client can 
download the file, bypass conversion, and import the binary directly.

Furthermore the .jbin files can be cached on the client just as other files are 
and the SHA-1 value in the datawrapper can verify the cached copy is up to date.

.obj files will never be discarded. They will continue to exist in the 
default-data svn and the d-d jar file. default-data will never contain JME 
binary files because these can be JME-version specific.

The server will have 2 cases:
(1) New server being deployed, the build process will
- convert .obj files to .jbin
- in the file's datawrapper
    - change the URL to reflect .jbin
    - change the type field to indicate it's JME binary
    - calculate and set the sha1 field to be the SHA-1 on the .jbin file
- proceed with server deployment

(2) Clients import new .obj files (PUT) or replace existing ones (POST)
- server receives the uploaded .obj file
- server converts it to .jbin
- server constructs and inserts into the database a datawrapper with the .jbin 
file's URL, SHA-1, and type
- server retains the .obj file for recycling into default-data

Original comment by davlit0...@gmail.com on 3 Sep 2010 at 8:53

• Added labels: Type-Enhancement
• Removed labels: Type-Other

[GoogleCodeExporter]

Let me argue in favor of having both the OBJ and the JBIN in the data wrapper.  
The purpose is mainly for other clients to be able to access the OBJ.  OBJ is 
an extremely open format and there is lots of support for dealing with it.  
Given that one of our purposes is to make the data in WBC more accessible, we 
want to allow users to download the OBJs that go along with the content they 
are viewing.  We can't implement a "download as OBJ" when a user clicks on a 
file if we remove the OBJ from the data wrapper.

Thoughts?

Original comment by stephen....@gmail.com on 5 Sep 2010 at 2:26

• Changed state: Accepted

[GoogleCodeExporter]

Original comment by caprea on 2 Oct 2010 at 12:46

• Changed state: Fixed