How to test on my own dataset?

Reed-yang commented 4 days ago

I saw issue on Deflow that eval on custum dateset needs groud truth flow. I'm very exciting that you have this self-supervised version of this model and wondering whether it is possible to inference scene flow using your pretrained weight on dataset: has frame point clouds, poses, and segmentation maybe, then producing flow results.

Kin-Zhang commented 3 days ago

I think the comment reply in the issue: https://github.com/KTH-RPL/DeFlow/issues/3#issuecomment-2128533742 it explains that for evaluation you always need ground truth to do that. While for inference proposal, you don't need ground truth.

The only step you need to do is (write your dataset class to read data) or (transfer your data to our h5py file format):

        group.create_dataset('lidar', data=pc.astype(np.float32))
        group.create_dataset('ground_mask', data=gm.astype(bool))
        group.create_dataset('pose', data=pose.astype(np.float32))
        if flow_0to1 is not None:
            # ground truth flow information
            group.create_dataset('flow', data=flow_0to1.astype(np.float32))
            group.create_dataset('flow_is_valid', data=flow_valid.astype(bool))
            group.create_dataset('flow_category_indices', data=flow_category.astype(np.uint8))
            group.create_dataset('ego_motion', data=ego_motion.astype(np.float32))

There are two examples how you can have a h5py file right now, check: https://github.com/KTH-RPL/SeFlow/tree/main/dataprocess

Reed-yang commented 2 days ago

btw, is it possible to test on dataset that has no valid ground_mask available? Or is there a convenient way to compute ground_height.npy & se2.json on my aggregated point cloud (from all frames)? thx in advance, :)

Kin-Zhang commented 1 day ago

You can directly try this python package to get ground_mask: https://github.com/Kin-Zhang/linefit/tree/master

Reed-yang commented 20 hours ago

One more question, I noticed that in process_log, pose and car_frame_pc are input for model inference. What coordinate system is car_frame_pc established on, Can you explain it in detail? In each frame, is car_frame_pc ego-centric or is it based on a unified first moment as the origin (world coor)? What should I do if my data is in world coor?

Kin-Zhang commented 7 hours ago

I think it's fine for both, pc is ego-centric and you will save pose for the world coor, and the first pose can be the original world coor. Here is an example dataloader I wrote one year ago which data is in world coor:

KTH-RPL / SeFlow

How to test on my own dataset? #3