search

open-forest-observatory / geograypher

Multiview Semantic Reasoning with Geospatial Data
BSD 3-Clause "New" or "Revised" License
10 stars 4 forks source link

Set camera frustum size intelligently in mesh vis #58

Closed russelldj closed 3 months ago

russelldj commented 5 months ago

Currently, the camera frustum size is set in unit of the internal coordinate system of the mesh. Instead, it should be set in meters, or relative to the scale of the mesh.

russelldj commented 4 months ago

This issue refers to how cameras are visualized. As seen below, each camera is represented by a pyramid, representing the viewing frustum. Setting how "tall" this pyramid is what we mean by the frustum scale, since this can be chosen arbitrarily. image (1)

Each individual camera frustum is visualized here. The documentation on the frustum scale says it's in "world units" but for cameras that come from metashape, this isn't quite accurate. Instead, this frustum scale is relative to an arbitrarily-scaled coordinate frame that is used for internal metashape processing.

@asidhu0 , I previously mentioned trying to set the scale inteligently in two places, but lets focus on one for now. This is when a camera set is added to the mesh, which is here. There is an option to provide a camera_set to be shown alongside the mesh. What I propose is to add a keyword that allows the user to set the the frustum scale in units of meters. Unfortunately, the PhotogrammetryCameraSet.vis method also takes a frustum scale in the units of the internal metashape coordinate system. So we need to find a scale factor between this coordinate system and one with known units, such as meters.

Here's where we need to do a little math. If mesh came from metashape, the self.local_to_epgs_4978_transform attribute should be set to a 4x4 matrix. If this isn't set, there is no transform and the scale factor is just 1. This matrix transforms a point in the mesh's coordinate local coordinate system to a point in the earth-centered earth-fixed coordinate system, EPGS:4978. This is a reference frame that has units of meters. You can determine the scale factor between these two coordinate systems by taking the cube root of the determinant of the upper-left 3x3 sub-matrix. Then you can divide the provided scale (in meters) by this scale factor, to get the frustum scale in the metashape coordinate frame.

I know this is dense, and I'm happy to answer any questions when you get into it.