Question about Anomaly Map Visualization for 3D AD&S

[JerryX1110]

Thanks for your insightful paper about 3D AD&S and the released neat code! I am trying to make some visualizations of the 3D data and blend an anomaly score map on it. However, I have a problem with the camera intrinsic parameter settings. Do you know how to visualize the point cloud with the given camera parameters using Open3D?

[wyzjack]

Same question here.

[mjack3]

Same, I would like to visualize the xyz image. Did you manage to do it?

[mujtabaasad]

was any able to visulize anomaly maps??

[lprintf]

The paper of Patchcore has some problem. We suggest that you pay attention to this problem, so as not to step on the pit again. see: https://www.webofscience.com/wos/woscc/full-record/WOS:001305143000001

[lprintf]

was any able to visulize anomaly maps??

I don't do research anymore, but I might be able to help.

[mujtabaasad]

was any able to visulize anomaly maps??

I don't do research anymore, but I might be able to help.

Can you please give some suggestions about visulization?? Thank you

[mujtabaasad]

The paper of Patchcore has some problem. We suggest that you pay attention to this problem, so as not to step on the pit again. see: https://www.webofscience.com/wos/woscc/full-record/WOS:001305143000001

What kind of problems???

[lprintf]

What kind of problems??? The Patchcore paper defines a module with side effects, and there is no ablation experiment, no code implementation, but it is implemented by this project. https://www.webofscience.com/wos/woscc/full-record/WOS:001305143000001, this paper discussed and solved the problem, are interested can read.

[lprintf]

was any able to visulize anomaly maps??

I don't do research anymore, but I might be able to help.

Can you please give some suggestions about visulization?? Thank you

I will help you find the code this weekend, too long haven't done this work, a little forgot. Of course, it is better that the author can open source.

[mujtabaasad]

was any able to visulize anomaly maps??

I don't do research anymore, but I might be able to help.

Can you please give some suggestions about visulization?? Thank you

I will help you find the code this weekend, too long haven't done this work, a little forgot. Of course, it is better that the author can open source.

Thank you... I will wait for your response. Appreciate it

[lprintf]

was any able to visulize anomaly maps??

I don't do research anymore, but I might be able to help.

Can you please give some suggestions about visulization?? Thank you

I will help you find the code this weekend, too long haven't done this work, a little forgot. Of course, it is better that the author can open source.

Thank you... I will wait for your response. Appreciate it

I'm sorry. I didn't manage my code well. The following I have written from memory, I hope it will be useful.

import numpy as np
import open3d as o3d
import matplotlib.pyplot as plt

# Function to load point cloud data
def load_point_cloud_data(point_cloud_file):
    """
    Load point cloud data from a .npy file.
    The file should contain a 3D tensor with shape (3, 224, 224).

    Args:
        point_cloud_file (str): Path to the .npy file containing the point cloud.

    Returns:
        np.ndarray: A (N, 3) array of points where N is the total number of points.
    """
    # Load the point cloud tensor
    point_cloud = np.load(point_cloud_file)
    # Flatten the x, y, z components
    x, y, z = point_cloud[0].flatten(), point_cloud[1].flatten(), point_cloud[2].flatten()
    # Combine them into a single array of 3D points
    points = np.vstack((x, y, z)).T
    return points

# Function to load RGB image data
def load_rgb_image(rgb_image_file):
    """
    Load RGB image data and map it to corresponding point cloud points.

    Args:
        rgb_image_file (str): Path to the .png file containing the RGB image.

    Returns:
        np.ndarray: A (N, 3) array of RGB values normalized to the range [0, 1].
    """
    # Read the RGB image
    rgb_image = plt.imread(rgb_image_file)
    # Flatten the image and normalize RGB values to [0, 1]
    return rgb_image.reshape(-1, 3) / 255.0

# Function to load precomputed outlier mask
def load_outlier_mask(outlier_mask_file):
    """
    Load the outlier mask from a .npy file.
    The file should contain a tensor with shape (1, 224, 224).

    Args:
        outlier_mask_file (str): Path to the .npy file containing the outlier mask.

    Returns:
        np.ndarray: A boolean array of shape (N,) indicating outlier points.
    """
    # Load the outlier mask tensor
    outlier_mask = np.load(outlier_mask_file)
    # Flatten and convert to a boolean array
    return outlier_mask.flatten().astype(bool)

# Function to visualize the point cloud
def visualize_point_cloud(points, colors, outliers):
    """
    Visualize the point cloud with normal points and outliers.

    Args:
        points (np.ndarray): A (N, 3) array of point cloud data.
        colors (np.ndarray): A (N, 3) array of RGB values for the points.
        outliers (np.ndarray): A boolean array indicating outlier points.
    """
    # Create an Open3D PointCloud object
    pcd = o3d.geometry.PointCloud()
    # Assign points and their colors
    pcd.points = o3d.utility.Vector3dVector(points)
    pcd.colors = o3d.utility.Vector3dVector(colors)

    # Separate normal points and outliers
    normal_pcd = pcd.select_by_index(np.where(~outliers)[0])
    outlier_pcd = pcd.select_by_index(np.where(outliers)[0])

    # Color outliers red
    outlier_pcd.paint_uniform_color([1, 0, 0])

    # Visualize the normal points and outliers together
    o3d.visualization.draw_geometries([normal_pcd, outlier_pcd])

# Main function
def main():
    """
    Main pipeline to:
    1. Load point cloud data.
    2. Load corresponding RGB image.
    3. Use the model-provided outlier mask.
    4. Visualize the results.
    """
    # File paths
    point_cloud_file = "0.tff"  # Path to the point cloud file (assumed .npy format)
    rgb_image_file = "0.png"    # Path to the RGB image file
    outlier_mask_file = "outlier_mask.npy"  # Path to the precomputed outlier mask

    # Load the point cloud data
    points = load_point_cloud_data(point_cloud_file)
    # Load the corresponding RGB image
    colors = load_rgb_image(rgb_image_file)
    # Load the precomputed outlier mask
    outliers = load_outlier_mask(outlier_mask_file)

    # Output statistics
    print(f"Total points: {len(points)}, Outliers: {np.sum(outliers)}")

    # Visualize the point cloud with outliers
    visualize_point_cloud(points, colors, outliers)

if __name__ == "__main__":
    main()