Closed INGnowait closed 2 years ago
问题我解决了,在./tools/lm/1_compute_fps.py,可以生成fps_points.pkl文件。然后开始训练,但是显示eta 3 days,要这么久吗==!用的2080ti。有什么可以加速的呢?
我也用的2080ti, 训练lm应该不到10小时。
Hello @wangg12,
Is the code for farthest point sampling in 'csrc' folder is written by you or borrowed from other repo? I am asking because I need some help in compiling the cpp extension on Windows machine.
I am getting following error:
Thank you, Supriya
It is from PVNet.
[10/07 16:39:18 fvcore.common.checkpoint]: No checkpoint found. Initializing model from scratch renamed 'output/gdrn/lm/a6_cPnP_lm13/tb/events.out.tfevents.1633595685.ing' -> 'output/gdrn/lm/a6_cPnP_lm13/tb_old/events.out.tfevents.1633595685.ing' WRN [1007_163918 core.gdrn_modeling.data_loader@172]: replace bg WRN [1007_163918 core.gdrn_modeling.data_loader@172]: replace bg [1007_163918 core.base_data_loader@263]: get bg image paths [1007_163918 core.base_data_loader@263]: get bg image paths [1007_163918 core.base_data_loader@274]: get bg_paths from cache file: .cache/bg_paths_VOC_table_4105f4eff7f37d64b73553e22f107499.pkl [1007_163918 core.base_data_loader@274]: get bg_paths from cache file: .cache/bg_paths_VOC_table_4105f4eff7f37d64b73553e22f107499.pkl [1007_163918 core.base_data_loader@276]: num bg imgs: 538 [1007_163918 core.base_data_loader@276]: num bg imgs: 538 WRN [1007_163919 core.gdrn_modeling.data_loader@172]: replace bg for real [1007_163919 core.base_data_loader@263]: get bg image paths [1007_163919 core.base_data_loader@274]: get bg_paths from cache file: .cache/bg_paths_VOC_table_4105f4eff7f37d64b73553e22f107499.pkl [1007_163919 core.base_data_loader@276]: num bg imgs: 538 [1007_163919 core.gdrn_modeling.engine@241]: Starting training from iteration 0 Traceback (most recent call last): File "core/gdrn_modeling/main_gdrn.py", line 159, in
args=(args,),
File "/home/j/anaconda3/envs/GDN/lib/python3.7/site-packages/detectron2/engine/launch.py", line 82, in launch
main_func(*args)
File "core/gdrn_modeling/main_gdrn.py", line 122, in main
do_train(cfg, args, model, optimizer, resume=args.resume)
File "/media/j/data/GDR-Net-main/core/gdrn_modeling/../../core/gdrn_modeling/engine.py", line 252, in do_train
data = next(data_loader_iter)
File "/home/j/anaconda3/envs/GDN/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in next
data = self._next_data()
File "/home/j/anaconda3/envs/GDN/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data
return self._process_data(data)
File "/home/j/anaconda3/envs/GDN/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
data.reraise()
File "/home/j/anaconda3/envs/GDN/lib/python3.7/site-packages/torch/_utils.py", line 428, in reraise
raise self.exc_type(msg)
AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/j/anaconda3/envs/GDN/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
data = fetcher.fetch(index)
File "/home/j/anaconda3/envs/GDN/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/j/anaconda3/envs/GDN/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/media/j/data/GDR-Net-main/core/gdrn_modeling/../../core/gdrn_modeling/data_loader.py", line 651, in getitem
processed_data = self.read_data(dataset_dict)
File "/media/j/data/GDR-Net-main/core/gdrn_modeling/../../core/gdrn_modeling/data_loader.py", line 537, in read_data
fps_points = self._get_fps_points(dataset_name)[roi_cls]
File "/media/j/data/GDR-Net-main/core/gdrn_modeling/../../core/gdrn_modeling/data_loader.py", line 204, in _get_fps_points
loaded_fps_points = data_ref.get_fps_points()
File "/media/j/data/GDR-Net-main/ref/lm_full.py", line 120, in get_fps_points
assert osp.exists(fps_points_path), fps_points_path
AssertionError: /media/j/data/GDR-Net-main/datasets/BOP_DATASETS/lm/models/fps_points.pkl
出现这两个报错:AssertionError: /media/j/data/GDR-Net-main/datasets/BOP_DATASETS/lm/models/fps_points.pkl 和AssertionError: Caught AssertionError in DataLoader worker process 0. 这是什么原因呢?lm数据集是从官方网站下载的,pytroch1.7+cuda10.2.谢谢