lbhqw
commented
1 month ago I meet the same errors! have you sloved ?
Open WLXM0922 opened 1 month ago
lbhqw
commented
1 month ago I meet the same errors! have you sloved ?
WLXM0922
commented
1 month ago I meet the same errors! have you sloved ?
Hello, I used to modify the parameter num_sample1, reducing it by 1000 each time, and then re-run it until it stopped reporting errors.
lbhqw
commented
1 month ago Will this not affect the subsequent model training? Can you train the model normally
WLXM0922
commented
1 month ago Will this not affect the subsequent model training? Can you train the model normally
Yeah, it has not affected the subsequent model training, but the trained model seems to have a poor effect. The average error of location recognition is over 40m, and the minimum error of feature matching (every 20 point pairs) is about 10m.
lbhqw
commented
1 month ago I can use other methods to regenerate the training set, but the following problems will occur during the training, it seems that there is a problem with the generated training set data, can you share the code you generated the data set。
：Begin training...
Traceback (most recent call last):
File "train.py", line 181, in
WLXM0922
commented
1 month ago I can use other methods to regenerate the training set, but the following problems will occur during the training, it seems that there is a problem with the generated training set data, can you share the code you generated the data set。 ：Begin training... Traceback (most recent call last): File "train.py", line 181, in train_and_valid(epoch) File "train.py", line 26, in train_and_valid for data in train_loader: File "/home/l/miniconda3/envs/pytorch-cuda111/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 631, in next data = self._next_data() File "/home/l/miniconda3/envs/pytorch-cuda111/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1346, in _next_data return self._process_data(data) File "/home/l/miniconda3/envs/pytorch-cuda111/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1372, in _process_data data.reraise() File "/home/l/miniconda3/envs/pytorch-cuda111/lib/python3.8/site-packages/torch/_utils.py", line 705, in reraise raise exception RuntimeError: Caught RuntimeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/l/miniconda3/envs/pytorch-cuda111/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop data = fetcher.fetch(index) # type: ignore[possibly-undefined] File "/home/l/miniconda3/envs/pytorch-cuda111/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/l/miniconda3/envs/pytorch-cuda111/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 51, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/l/miniconda3/envs/pytorch-cuda111/lib/python3.8/site-packages/torch_geometric/data/dataset.py", line 289, in getitem data = self.get(self.indices()[idx]) File "/home/l/bathy_nn_learning/dataset.py", line 66, in get data_src, center_src = self.submap_set[idx_src] File "/home/l/miniconda3/envs/pytorch-cuda111/lib/python3.8/site-packages/torch_geometric/data/dataset.py", line 289, in getitem data = self.get(self.indices()[idx]) File "/home/l/bathy_nn_learning/dataset.py", line 41, in get raise RuntimeError(f"Empty center at index {idx}") RuntimeError: Empty center at index 1022
Didn't you report an error when running parse_cereal.py? According to your question, it seems that the generated training set is incomplete, but I haven't changed the code for generating the training set, only the parameter mentioned before. In addition, during training, I changed the parameter BATCH_SIZE to 8.
lbhqw
commented
1 month ago I re-cloned the library and when executed running parse_cereal.py:(pytorch-cuda111) l@l:~/bathy_nn_learning$ python scripts/parse_cereal.py [Open3D WARNING] Write geometry::PointCloud failed: unknown file extension bin for file data/train_trajectory.bin. [Open3D WARNING] Write geometry::PointCloud failed: unknown file extension bin for file data/test_trajectory.bin. have you meeted the same error?
WLXM0922
commented
1 month ago I re-cloned the library and when executed running parse_cereal.py:(pytorch-cuda111) l@l:~/bathy_nn_learning$ python scripts/parse_cereal.py [Open3D WARNING] Write geometry::PointCloud failed: unknown file extension bin for file data/train_trajectory.bin. [Open3D WARNING] Write geometry::PointCloud failed: unknown file extension bin for file data/test_trajectory.bin. have you meeted the same error?
Yeah, I have also encountered this error. I directly changed the suffix of these two files to ".pcd", which can also generate data. You can try this method.
WLXM0922
commented
1 month ago Then, in the utils/data.py file, you need to add a function to read the ".pcd" file (the following is the read_bin_plus function I defined), as follows: def read_bin_plus(file_name, dtype=None): """ Read .pcd files and get xyz points """ if not dtype: dtype = np.float32
""" Read pcd file with Open3D library """ pcd = o3d.io.read_point_cloud(file_name)
""" Convert point cloud data into NumPy array """ pcd_np = np.asarray(pcd.points) return pcd_np
Then, in the generate_test_set.py file, you need to change all "read_bin" to "read_bin_plus". If I remember correctly, there are three places to change.
lbhqw
commented
1 month ago Thank you very much for your patience! I'll try again.
lbhqw
commented
1 month ago Hello, I have listened to your advice, but the following problems occurred when executing the training script, may I ask if you encountered them at that time?Begin training...
Traceback (most recent call last):
File "train.py", line 183, in
WLXM0922
commented
1 month ago Sorry, I haven't encountered this mistake. Judging from the information you gave, it may be that the training set was generated incorrectly.
Hello, thank you very much for this job. When I run the generate_training_set.py file to generate a training set, I get an error: Traceback (most recent call last): File "bathy_nn_learning/scripts/generate_training_set.py", line 85, in
processed_bin = random_sample(raw_bin, num_sample1)
File "bathy_nn_learning/utils/dataset.py", line 58, in random_sample
pcd_np_sample = pcd_np[np.random.choice(pcd_np.shape[0], size=int(n_points), replace=False), :]
File "mtrand.pyx", line 984, in numpy.random.mtrand.RandomState.choice
ValueError: Cannot take a larger sample than population when 'replace=False' I thought the size of random sampling was too large, so I changed the parameter num_sample1 = 19000 to a smaller size, but after re-running the code, num of points: min will also be smaller, which is still the above error.
Why is this? Looking forward to your answer, thank you very much.