Visualizer custom settings

[aymanhab]

The following changes to default visualization lead to better visuals (annotated for ease of implementation, impact smaller is better)

1. Adjust the camera's field of view (particularly helpful for images in sagittal plane) (1, 1)
2. Adjust the near and far clipping planes so model is rendered at maximum resolution (ideally, use separate rendering pass for environment) (3, 1)
3. Change the color/intensity/position of the sunlight, ambient light, and camera light (1, 1)
4. Add a faint light beneath the model (color depends on floor color) to model reflection from sunlight (1, 3)
5. Change floor material to remove shininess (1, 4)
6. Change the color of the bones (3, 1)
7. Compute texture coordinates and texture map the bones (may cause GUI slowdown/crash) (4, 1)
8. Fix the marker lighting (https://github.com/opensim-org/opensim-gui/issues/1107) (1, 2)
9. Allow users to export images at much higher resolution (?, 2)
10. Increase shadow frustum size and shadow map depth to reduce rendering artifacts (3, 3)

[aymanhab]

@jenhicks Please let me know if/what part you think we should include in 4.1

[aymanhab]

@tkuchida I added annotation to indicate difficulty, impact. You're in a better position to judge difficulty/impact since you've been on both sides @tkuchida so please feel free to chime in.

[tkuchida]

Thanks @aymanhab. A few questions: (a) How do you plan to implement these? Knobs on the "View Controls" panel? (b) Some of these would be easy to do but difficult to do well. For example, IIRC changing the camera's field of view requires a GUI restart (can't remember why, but might have had to do with the clipping planes). Automatically updating clipping planes would take some code to do right and is necessary to avoid large reductions in resolution. We could expose these to the user, but would then need to explain what they are and how to adjust them. (c) If we add more View Controls, I think we should implement saving/loading these variables. For example, a user who spends 10 minutes tweaking lighting will want to save these settings and load them next time.

[jenhicks]

Thanks @aymanhab and @tkuchida . As far as what we might include in 4.1, I think the scope would be anything that involves changing a default setting/parameter that would lead to visuals that are better for most users.

For changes that are more difficult or would require tweaking on a user-by-user basis, I think this would be longer term so that we could develop a nice interface for setting and saving the settings (and perhaps could provide a couple different setting configurations for common use-cases).

@aymanhab from the point system, I don't have a good sense still of what would meet the criteria I describe in the first paragraph. Could you add some more detail? Or I'm happy to meet and discuss in person.

[aymanhab]

@tkuchida I don't think there's a single place that's a good fit for all items on this list:

• Object specific changes (e.g. Bone-texture) belong to model files
• Some settings e.g. related to lighting, floor, and Camera belong to the View Controls or the Toolbar
• Other settings (e.g. those that don't take effect immediately and require a restart) belong to a more permanent storage medium (e.g. files that get read on launch), building a GUI for them while not interactive in nature would be of limited use.

I'd lean to have a file with a list of "presets" that include camera parameters, lights, etc. and users have a persistent preference that selects among those presets, with a special "custom" preset that would allow expert users to replicate settings with minimum effort,rather than have a sophisticated GUI that gets used/exercised by a very slim fraction of users. Sure would love to hear your thoughts as our most expert user in this area as well.

[tkuchida]

Not sure that would be the best way to split up settings. For example, if the user can specify in the model file that a mesh is metal or bone, then the GUI could provide an interface to adjust the visualization parameters for all meshes of a given material. So the "object-specific changes" would be implemented partly in the model file and partly in the GUI. Also, there are settings that currently require a restart, but it may be possible to update these live if the right things are updated programmatically.

I think lighting would be a good one to start with. Some questions:

• How will users save/load lighting settings?
• Should the user be able to lock the sunlight to the model so that the lighting conditions remain the same as the model moves around the virtual lab?
• If users can adjust the location of the sunlight, we should recompute near and far clipping planes of the "camera" that is used to cast shadows. Otherwise, shadows will get clipped off and disappear as the sunlight is moved away from the model.

[aymanhab]

@adamkewley This is an old issue that has a summary of observations/recommendations regarding the visualizer by @tkuchida. Clipping planes are mentioned here so would be good to consolidate the input here and the latest feedback on 4.4 Beta.

[aymanhab]

@tkuchida I'm getting back to this since the web-viewer could have some preset environments and we could bake in choices for builtin models. I vaguely remember you had a custom javascript file to customize settings for gait models. It has been few years though so you may not have these anymore. If you do, we should touch base so we're not reinventing the wheel. Thank you

[tkuchida]

Hi @aymanhab! David and I found that each model, pose, and camera angle required tweaking to get something looking good. Very much like a photographer in a studio. The key "knobs" were adjusting near and far clipping planes to maximize the resolution of the shadow map, adjusting the model light position/orientation/attenuation, and adjusting the floor light (a light blue color pointing up from the floor). Happy to chat about it 🗨️😄