Spike: what do we need to view 3D models

ShanaLMoore commented 1 year ago

Will this require an upgrade to 4? What do we need for iiif to view 3d models? reference manifests as examples - put into viewer as is to see what happens?

We will then need to be able to generate this manifest.

We may or may not need to upgrade the viewer.

What does uploading a 3d file look like? How does fits figure this out?

ref #196

jillpe commented 1 year ago

3D model example: https://doi.org/10.23636/1128 3D models for testing: https://drive.google.com/drive/folders/1rOrkz-R8jXKDZ-I9Z39WO20iDmCLBJFf

ShanaLMoore commented 1 year ago

3d Example per Kirk: https://www.morphosource.org/concern/media/000158446?locale=en manifest: https://www.morphosource.org/manifests/ec00bb70-e34a-4d34-be5a-8d6bae7b4ffe.json github: https://github.com/MorphoSource forked iiif manifest: https://github.com/MorphoSource/iiif_manifest

kirkkwang commented 1 year ago

Formats

Looks like UV should support both .glb and gltf formats https://github.com/UniversalViewer/universalviewer/blob/main/src/content-handlers/iiif/IIIFContentHandler.ts#L139-L140 https://github.com/IIIF-Commons/vocabulary/blob/master/src/index.ts#L69-L70

From this thread in UV channel, Ed is recommending using .glb because it's easier to link to within the manifest

Generic manifest example

https://gist.github.com/edsilv/2228d75d3a912d4c5fb2099bfc69d852

Characterization

MorphoSouce seems to use this Python script for characterizing. Mime types seem to be based on file extensions. https://github.com/MorphoSource/blender_characterize_mesh/blob/master/blender_characterize_mesh.py#L64-L71

Misc

How to download obj from Sketchfab https://www.youtube.com/watch?v=v3_e2dW2ECE as referenced by Ed in the screenshot
I've reached out to Julie Winchester from Duke through Samvera Slack to see if I can ask a few questions. Julie is the lead dev and product manager for MorphoSource.
I've reached out to Scott Mcavoy from UCSD who is a old coworker of mine that is involved in the IIIF 3D community.

ShanaLMoore commented 1 year ago

https://ms1.morphosource.org/About/launch

Data Integrity and Standards

Digital Preservation: A primary reason for the refactor was to upgrade MorphoSource from a custom-written LAMP stack web application to a digital preservation stack. MorphoSource 2.0 is a Rails-based customized [Hyrax](https://hyrax.samvera.org/) instance. Hyrax is an open source platform widely adopted by digital repositories around the world and thus has an active community of developers maintaining and advancing it. At launch, MorphoSource 2.0 will use Fedora as the digital asset management layer and Solr for efficient querying at scale, allowing for steady future repository growth.

IIIF and Universal Viewer: The [International Image Interoperability Framework](https://iiif.io/) (IIIF) is an API and set of standards for enabling standardized display and transmission of documents and media on the web and in other domains. [Universal Viewer](http://universalviewer.io/) (UV) is a popular open source browser viewer that relies on IIIF, and is supported by Hyrax by default. MorphoSource has worked with Ed Silverton of [Mnemoscene](https://mnemoscene.io/) to create [Aleph](https://github.com/aleph-viewer/aleph), an extension to the UV to enable 3D volume viewing, 3D annotations and measurements, and more nuanced viewing of meshes, including colored/textured surface meshes. Aleph is built on an [A-Frame](https://aframe.io/) foundation, and so will soon be VR-capable. As Aleph and Universal Viewer are both open source, this technology is available for use by other teams and other Hyrax repositories in particular. We intend to continue collaboratively developing this viewer for the benefit of MorphoSource users and the larger community

kirkkwang commented 1 year ago

[ ] Upgrade UV to 4.0.17
- In order to render the 3D object, we need to upgrade Universal Viewer to 4+
[ ] Extend IIIF Manifest generation to handle GLB files
- GLB is the recommended format from Ed Silverton for presenting in UV
[ ] Ensure the UV shows a view file link for a work that has a 3D model
[ ] Create a GLB file
- The example given to us was a nested set of zip files and within a directory there were three files that represent the 3D object. We need to script the process of converting the three files into a glb format
- CAVEATS: How do we know which three files go together? We know the three extensions we're looking for, but with the data we had we had layers of zips, so finding the right files is something to consider. We assume zip is in ./source and within that zip file we have three files all with the same base name but with the following extentions: obj, mtl, and jpg.
[ ] Determine the sh command that blends the obj, mtl, and jpg files
[ ] Add blender and dependencies to the build
[ ] Build and incorporate derivative generation into Hyrax::DerivativeService stack
[ ] Spike: Sniff zip files contents to see if it is a 3D model (NOTE: there is as 3D Image Resource Type)

Proof of concept video

https://user-images.githubusercontent.com/19597776/222216470-bd31568a-f1b4-4315-80ba-26631420568d.mp4

kirkkwang commented 1 year ago

https://github.khronos.org/glTF-Tutorials/BlenderGltfConverter/ docs: https://docs.blender.org/manual/en/latest/advanced/command_line/render.html

This script can programmatically convert .obj files to .glb files. Just need Blender installed. I think Blender has Python installed within it along with any dependencies it would need.

The glTF and obj files from the 3D example seem to be downloaded from SketchFab. The three files we need to create a .glb would be the obj, mtl, and jpg all in the same directory.

There's something called "Draco mesh compression" and it's for optimizing for the 3D viewer, smaller == faster transfer rates. But just like with jpeg compression, it can be a little too much. Might need to look into this

scientist-softserv / britishlibrary