ValueError: buffer size must be a multiple of element size

[mostafa501]

1 Hi Dr. @filaPro, thank you for your previous help, we successfully run the demo test file, all this through using scannet data file (/demo/data/scannet/scene0000_00.bin). And using this snippet: " from argparse import ArgumentParser from mmdet3d.apis import inference_detector, init_model from mmdet3d.apis import show_result_meshlab parser = ArgumentParser() parser.add_argument('--pcd', default='/media/navlab/GNSS/work/detection/tr3d-main/demo/data/our_data/used_chair.bin', help='Point cloud file') #'/data/scannet/scene0000_00.bin' parser.add_argument('--config', default='/media/navlab/GNSS/work/detection/tr3d-main/tr3d/configs/tr3d/tr3d_scannet-3d-18class.py', help='Config file') # tr3d_scannet-3d-18class.py # tr3d_scannet.pth'
parser.add_argument('--checkpoint', default='/media/navlab/GNSS/work/detection/tr3d-main/pretrained_models/3d_detetction/scannet/tr3d_scannet.pth', help='Checkpoint file') parser.add_argument('--device', default='cuda:0', help='Device used for inference') parser.add_argument('--score-thr', type=float, default=0.3, help='bbox score threshold') parser.add_argument('--out-dir', type=str, default='/media/navlab/GNSS/work/detection/tr3d-main/evaluate_results/try1', help='dir to save results') parser.add_argument('--show',action='store_true',help='show online visualization results') parser.add_argument('--snapshot',action='store_true',help='whether to save online visualization results') args = parser.parse_args() model = init_model(args.config, args.checkpoint, device=args.device) result, data = inference_detector(model, args.pcd) print (result) show_result_meshlab(data,result,args.out_dir,args.score_thr,show=args.show,snapshot=args.snapshot,task='det') "

---

2 But when using the same script with my own data with same extension of bin file " used_chair.bin", we got this error: " load checkpoint from local path: /media/navlab/GNSS/work/detection/tr3d-main/pretrained_models/3d_detetction/scannet/tr3d_scannet.pth Traceback (most recent call last): File "/media/navlab/GNSS/work/detection/tr3d-main/demo/pcd_demo_used_ok.py", line 19, in result, data = inference_detector(model, args.pcd) File "/media/navlab/GNSS/work/detection/tr3d/mmdet3d/apis/inference.py", line 140, in inference_detector data = test_pipeline(data) File "/media/navlab/GNSS/work/detection/tr3d/mmdet3d/datasets/pipelines/compose.py", line 49, in call data = t(data) File "/media/navlab/GNSS/work/detection/tr3d/mmdet3d/datasets/pipelines/loading.py", line 486, in call points = self._load_points(pts_filename) File "/media/navlab/GNSS/work/detection/tr3d/mmdet3d/datasets/pipelines/loading.py", line 463, in _load_points points = np.frombuffer(pts_bytes, dtype=np.float32) ValueError: buffer size must be a multiple of element size"

---

3 attached is the file we trying to use it: used_chair.zip We got this error with any other file although these files correctly contain data.

Can you please help me to solve this error or adjust my used data format to correctly run the script? Thank you.

[filaPro]

ValueError: buffer size must be a multiple of element size"

This simply means that your .bin file can not be loaded as np.float32, probably because it was not saved with dtype=np.float32.

[mostafa501]

1 Thank you dr @filaPro. We overcome this problem by converting test file to be saved as dtype=np.float32. Now, we run this script to test owr own data: "" from argparse import ArgumentParser from mmdet3d.apis import inference_detector, init_model from mmdet3d.apis import show_result_meshlab parser = ArgumentParser() parser.add_argument('--pcd', default='/media/navlab/GNSS/work/detection/tr3d-main/demo/data/our_data/used_chair_ok.txt', help='Point cloud file')
parser.add_argument('--config', default='/media/navlab/GNSS/work/detection/tr3d-main/tr3d/configs/tr3d/tr3d_scannet-3d-18class.py', help='Config file') # tr3d_scannet-3d-18class.py
parser.add_argument('--checkpoint', default='/media/navlab/GNSS/work/detection/tr3d-main/pretrained_models/3d_detetction/scannet/tr3d_scannet.pth', help='Checkpoint file') parser.add_argument('--device', default='cuda:0', help='Device used for inference') parser.add_argument('--score-thr', type=float, default=0.3, help='bbox score threshold') parser.add_argument('--out-dir', type=str, default='/media/navlab/GNSS/work/detection/tr3d-main/evaluate_results/try1', help='dir to save results') parser.add_argument('--show',action='store_true',help='show online visualization results') parser.add_argument('--snapshot',action='store_true',help='whether to save online visualization results') args = parser.parse_args() model = init_model(args.config, args.checkpoint, device=args.device) result, data = inference_detector(model, args.pcd) print (result)

with open(args.pcd, 'r') as file: lines = file.readlines() num_rows = len(lines) first_line_values = lines[0].split(',') num_columns = len(first_line_values) print(f"datasize: Rows-{num_rows},Columns-{num_columns}" "" We follow this issue to adjust format of tested file: https://github.com/SamsungLabs/tr3d/issues/8 the used file composed of (x,y,z,r,g,b) and we used txt file as pcd file sometimes saved as (x,y,z) only. Here is the file used: used_chair_ok.txt

---

2 However, we got another error with any file used, although we are sure that the file has 6 columns as printed: "" datasize: Rows-21862,Columns-6

load checkpoint from local path: /media/navlab/GNSS/work/detection/tr3d-main/pretrained_models/3d_detetction/scannet/tr3d_scannet.pth Traceback (most recent call last): File "/media/navlab/GNSS/work/detection/tr3d-main/demo/pcd_demo_used_ok.py", line 58, in result, data = inference_detector(model, args.pcd) File "/media/navlab/GNSS/work/detection/tr3d/mmdet3d/apis/inference.py", line 140, in inference_detector data = test_pipeline(data) File "/media/navlab/GNSS/work/detection/tr3d/mmdet3d/datasets/pipelines/compose.py", line 49, in call data = t(data) File "/media/navlab/GNSS/work/detection/tr3d/mmdet3d/datasets/pipelines/loading.py", line 487, in call points = points.reshape(-1, self.load_dim) ValueError: cannot reshape array of size 392923 into shape (6) "" Can you please help me how to solve this error or adjust my used data format to correctly run the script?

[filaPro]

You can simply print points array just before reshape and see why its size can not be divided by 6.

[mostafa501]

Dr.@filaPro, here is the last error, we are sorry for distirbance again. We see, the insterted file is with datasize: Rows-21862,Columns-6, as uploaded, we used this line as you advised:
print(f"Loaded points size: {points.shape}") and got: Loaded points size: (392923,). we want to understand why this problem occur although we uploaded (x,y,z,r,g,b) file with dimension of 21862*6, unfortunately, we dont know where this number (392923) come from!.

---

Also we tried to make our data bin format like the used demo file (/demo/data/scannet/scene0000_00.bin), but there is something we cannot understand as when we load this file to cloud compre (its size is 954KB), we found that it has very very large number of points as in image, and can not be uploaded: Please help us to overcome this limitation, and tell us how this demo bin file is formed to be followed as this is the last step to benefit from the code.

[filaPro]

You can simply print points array just before reshape and see why its size can not be divided by 6.

I'm still recommending you to start with this. The situation looks like you have different array before np.save and after np.frombuffer. And it can be easily debugged on your side.

[mostafa501]

Ok, we will try again, thank you dr @filaPro. We just ask, is this network can be used for instance segmentation, as we noticed that it is similar to TD3D network?, and if right, how to be used for instance segmentation. Thank you again.

[filaPro]

No, for instance segmentation please use TD3D repo. And yes, its object detection part is based on TR3D.