Feature: 3D Renderer and Preview Export of Thumbnails

[rahelarnold98]

This pull request integrates a 3D model renderer and introduces a 3D model preview exporter.

The renderer, originally developed by @net-cscience-raphael for Cineast using LWJGL, retains its core functionality while the texture model has been translated to Kotlin for enhanced integration. This updated version replaces the previous implementation in the vitrivr-engine-core module.

In addition, new functionality has been added to create previews of 3D models in the vitrivr-engine-module-m3d/src/main/kotlin/org/vitrivr/engine/model3d/ModelPreviewExporter.kt. By specifying settings in the schema-config.json file, users can export either GIFs or JPGs that showcase 3D models. GIFs allow for a customizable number of views capturing different angles of the 3D model, while the JPG version provides four predefined views. To ensure correct formatting of exported previe ws, the mimeType of the DiskResolver must be adjusted in the config file.

Overview of new Functionality:

• Integration of a 3D model renderer
• Export of GIF previews for 3D models
• Export of JPG previews for 3D models

[ppanopticon]

Thanks for the PR.

In order to be able to review this it would be nice if you could add some minimal working example / documentation snippet as to how this can be used and tested.

Furthermore, I was wondering if it would be possible to have a minimal UnitTest that makes sure that rendering works. See PR #84 for further reference.

[ppanopticon]

In addition to the detailed comments and the previous remark, I have two more fundamental issue with this PR and the way the M3D module is setup:

1. I don't think it's particularly wise to add Java code to a Kotlin project, unless there is a good reason to (which I don't see).
2. I was wondering if the JOML dependency in the vitrivr-engine-core module could be removed by using our own datatypes and convert them, once Vector3f are actually needed.

In its current state, the PR seems to violate some basic principles of the project (e.g., the ability to include or omit certain modules). Furthermore, it seems to be a bit messy since it's littered with empty classes and deprecated code.

[rahelarnold98]

Thank you for the detailed review @lucaro. I will work through the comments and make the adjustments.

Nevertheless, I think we should discuss whether the complete rendering logic that uses LWJGL must be translated to Kotlin or if we can include this independent Java code. The code was directly copied from Cineast, so the finite state machine was implemented like this before.

Furthermore, I have not found a Java class in src/main/kotlin. Please point them out. For me, all Java code is located under src/main/java.

@ppanopticon Thank you as well for your feedback. I will add an example and unit tests and rewrite some parts to remove the JOML dependency in the core module.

[lucaro]

I'm aware that this was implemented like this in Cineast. It was over-engineered there (as I already pointed out back then), and it is over-engineered here. Here, it's even worse since Cineast had already collected some bloat over the years, so the addition was relatively not as bad. I'm also opposed to supporting multiple languages in a project with no clear benefit. This just complicates maintenance and reduces readability in the long run. Translating the Java code to Kotlin is straightforward and even (mostly) automated by Intellij. While translating this (or, mostly, having it automatically translated), there will also be room to get rid of some inherent boilerplate code that is just not required in Kotlin. You are right about the package structure, the Java classes all appear to be in src/main/java, my bad.

[ppanopticon]

Okay so I have taken another look at this, especially how this can be integrated without interfering with the flow of the integrating application. Having spent half a day on this I can conclude, that the only way we can use the rendering logic without hooking it into the application's main thread is by externalising it into a different process.

For this purpose, I have created the Renderer and the ExternalRenderer implementation, which are basically two sides of a coin: The Renderer is a small Java application (basically a main() method) that reads RenderRequests from a an ObjectInputStream, processes them and writes them to the ObjectOutputStream. The ExternalRenderer spawns such a process, generates and sendsRenderRequests and processes incoming RenderResponses. The very simple example seems to work. However, there is an issue in ModelPreviewExporter.renderPreviewJPEG that makes the unit test fail (I'll add a comment).

Having said that: I agree with @lucaro that the entire rendering logic (i.e., this state-machine stuff) is way too complicated for solving a simple problem such as rendering a few previews. I really wonder, if this can't be simplified. To me, the fact that the entire state-machine logic in Worker is basically a Runnable (which is typically supposed to run in a different thread) and that GLFW must be executed on the main thread is a clear sign that the solution and the problem do not fit together.

[rahelarnold98]

The JOML dependency in the vitriol-engine-core module is now removed. Concerning documentation, I will add the instructions in the Wiki.

[rahelarnold98]

An example of how the configuration should look like to create previews can already be found in the Wiki: https://github.com/vitrivr/vitrivr-engine/wiki/Example#3d-model-pipeline

[rahelarnold98]

@lucaro This pull request is ready to merge from my side. Could you please review it once more?

[lucaro]

If @ppanopticon gave his approval, it's good enough for me 😉