Python APIs - wrong manual?

[SamueleBumbaca]

Main problem

Metashape_2.1.3 Metashape Python Reference (agisoft.com) module manual describes class Metashape.PointCloud.Points (pag 121) as a list, instead I found it to be 'type'. This means that is not iterable and the points are not written in. Where can I access individual point of the point cloud?

in context I want to segment the point cloud (and mesh vertices) by aligned and semantically segmented photos. As shown in https://github.com/JordanMakesMaps/3D-Model-Classification/tree/main, one can colorize the point cloud (or the model) by colorize dense points in GUI, but how to do that in Python?

I tryied in this way: assign the classes to the points of the point cloud as the final step of semantically segmenting the pictures of the alignment and mapping the segmentation on the point cloud by projecting the image coordinates in the point cloud coordinates.

import Metashape
from PIL import Image

# Function to assign labels from segmented images to the 3D model
def map_labels_to_mesh(chunk):
    # Ensure chunk contains aligned cameras and a 3D model
    assert chunk.cameras and chunk.point_cloud
    pc = chunk.point_cloud  # The dense point cloud

    # Loop through all cameras
    for camera in chunk.cameras:
        if not camera.enabled or not camera.transform:
                continue  # Skip disabled or non-aligned cameras
        # Get the camera's corresponding semantically segmented image
        label_image = Image.open('image/path')

        # Loop over all faces in the model
        for point in pc.Points:
# AS IN PAGE 121 Metashape.PointCloud.Points
            # Get point coordinates in world space
            coord = point.position # AS IN PAGE 121 Metashape.PointCloud.Point.position
            # Project 3D point to the image plane of the camera
            image_coords = camera.project(coord) # AS IN PAGE 26
            if 0 <= image_coords.x < camera.sensor.width and 0 <= image_coords.y < camera.sensor.height:
                # Map the pixel coordinate from the label image
                pixel_value = label_image.getpixel((int(image_coords.x), int(image_coords.y)))
                # Assign the label from the segmented image to the point
                point.classification = pixel_value

    return pc

but I cannot iterate on pc.Points (TypeError: 'type' object is not iterable) as pc.Points.class == <class 'type'>

[PolarNick239]

Hi,

Metashape_2.1.3 Metashape Python Reference (agisoft.com) module manual describes class Metashape.PointCloud.Points (pag 121) as a list, instead I found it to be 'type'. This means that is not iterable and the points are not written in. Where can I access individual point of the point cloud?

You mean that you find unexpected that type(Metashape.PointCloud.Points) prints type instead of list? This is because this is a name of a class, so Points is a type just like any other class (including list). For example, if you execute type(list) - you will also get type as a result:

[PolarNick239]

About your use-case (as my colleague replied):

Currently there is no direct way to interact with the dense point cloud points, so please check, how it could be done by exporting the point and then treating it as o3d/np arrays: https://github.com/agisoft-llc/metashape-scripts/blob/master/src/align_model_to_model.py

Alternatively, you can use PointCloud.Reader() class:

reader = Metashape.PointCloud.Reader()  
reader.open(chunk.point_cloud)
points = reader.read(10000)

In such manner you can access the points of the dense point cloud by blocks (like 10000 in the given example), as the complete point cloud could be too large to be loaded to the variable at once. And here you will have points of Metashape.PointCloud.Points class and each element of this array would be of Metashape.PointCloud.Point class.

[PolarNick239]

Also keep in mind that processing large numbers (>million) of elements (points) on per-element-basis is very slow in Python.

[SamueleBumbaca]

I really appreciate your and your colleague's help and I apologise for the misleading.

For the use case I found ColorizePointCloud or ColorizeModel as solution. For example, on mesh, you can colour the model with the segmentation binary mask. This will result in a mesh with vertices and faces colours somehow equal to the probability of being of the segmentation class.

import Metashape
def export_class_color_mesh(label):
    doc = Metashape.Document()
    doc.open('')
    mask_path_list = ''
    chunk = doc.chunk
    # Get the cameras
    cameras = chunk.cameras
    for camera, mask_path in zip(cameras, mask_path_list):
        camera.photo.path = mask_path
    colorizeModel = Metashape.Tasks.ColorizeModel()
    colorizeModel.source_data = Metashape.DataSource.ImagesData
    colorizeModel.apply(chunk)
    chunk.exportModel('')