open-mmlab / mmengine

OpenMMLab Foundational Library for Training Deep Learning Models
https://mmengine.readthedocs.io/
Apache License 2.0
1.18k stars 355 forks source link

[Bug] #1372

Open yangff0016 opened 1 year ago

yangff0016 commented 1 year ago

Prerequisite

Environment


UnicodeDecodeError Traceback (most recent call last) Cell In[11], line 6 3 from mmengine.config import Config 4 from mmengine.registry import init_default_scope ----> 6 cfg = Config.fromfile('rtmdet_tiny_1xb12-40e_cat.py') 8 init_default_scope(cfg.get('default_scope', 'mmdet')) 10 dataset = DATASETS.build(cfg.train_dataloader.dataset)

File ~\anaconda3\lib\site-packages\mmengine\config\config.py:178, in Config.fromfile(filename, use_predefined_variables, import_custom_modules, use_environment_variables) 165 """Build a Config instance from config file. 166 167 Args: (...) 175 Config: Config instance built from config file. 176 """ 177 filename = str(filename) if isinstance(filename, Path) else filename --> 178 cfg_dict, cfg_text, env_variables = Config._file2dict( 179 filename, use_predefined_variables, use_environment_variables) 180 if import_custom_modules and cfg_dict.get('custom_imports', None): 181 try:

File ~\anaconda3\lib\site-packages\mmengine\config\config.py:471, in Config._file2dict(filename, use_predefined_variables, use_environment_variables) 469 # Substitute predefined variables 470 if use_predefined_variables: --> 471 Config._substitute_predefined_vars(filename, 472 temp_config_file.name) 473 else: 474 shutil.copyfile(filename, temp_config_file.name)

File ~\anaconda3\lib\site-packages\mmengine\config\config.py:301, in Config._substitute_predefined_vars(filename, temp_config_name) 295 support_templates = dict( 296 fileDirname=file_dirname, 297 fileBasename=file_basename, 298 fileBasenameNoExtension=file_basename_no_extension, 299 fileExtname=file_extname) 300 with open(filename, encoding='utf-8') as f: --> 301 config_file = f.read() 302 for key, value in support_templates.items(): 303 regexp = r'{{\s' + str(key) + r'\s}}'

File ~\anaconda3\lib\codecs.py:322, in BufferedIncrementalDecoder.decode(self, input, final) 319 def decode(self, input, final=False): 320 # decode input (taking the buffer into account) 321 data = self.buffer + input --> 322 (result, consumed) = self._buffer_decode(data, self.errors, final) 323 # keep undecoded input until the next call 324 self.buffer = data[consumed:]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 96: invalid start byte

Reproduces the problem - code sample

-- coding: UTF-8--

from mmdet.registry import DATASETS, VISUALIZERS from mmengine.config import Config from mmengine.registry import init_default_scope

cfg = Config.fromfile('rtmdet_tiny_1xb12-40e_cat.py')

init_default_scope(cfg.get('default_scope', 'mmdet'))

dataset = DATASETS.build(cfg.train_dataloader.dataset) visualizer = VISUALIZERS.build(cfg.visualizer) visualizer.dataset_meta = dataset.metainfo

plt.figure(figsize=(16, 5))

只可视化前 8 张图片

for i in range(8): item=dataset[i]

img = item['inputs'].permute(1, 2, 0).numpy() data_sample = item['data_samples'].numpy() gt_instances = data_sample.gt_instances img_path = osp.basename(item['data_samples'].img_path)

gt_bboxes = gt_instances.get('bboxes', None) gt_instances.bboxes = gt_bboxes.tensor data_sample.gt_instances = gt_instances

visualizer.add_datasample( osp.basename(img_path), img, data_sample, draw_pred=False, show=False) drawed_image=visualizer.get_image()

plt.subplot(2, 4, i+1) plt.imshow(drawed_image[..., [2, 1, 0]]) plt.title(f"{osp.basename(img_path)}") plt.xticks([]) plt.yticks([])

plt.tight_layout()

Reproduces the problem - command or script

run

Reproduces the problem - error message

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 96: invalid start byte

Additional information

1.如何解决? 2.为什么报错?

yangff0016 commented 1 year ago

sys.platform: win32 Python: 3.10.9 | packaged by Anaconda, Inc. | (main, Mar 1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)] CUDA available: True numpy_random_seed: 2147483648 GPU 0: NVIDIA GeForce RTX 3080 Laptop GPU CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8 NVCC: Cuda compilation tools, release 11.8, V11.8.89 MSVC: n/a, reason: fileno PyTorch: 2.0.1+cu118 PyTorch compiling details: PyTorch built with:

TorchVision: 0.15.2+cu118 OpenCV: 4.8.0 MMEngine: 0.7.4 MMDetection: 3.0.0+unknown

collinmccarthy commented 3 months ago

The issue here for me was that the rank 0 GPU was not returning from the forward pass, and for whatever reason the broadcast wasn't blocking. So the returned tmpdir was a bunch of garbage characters some of which couldn't be decoded.

mike-egg123 commented 1 month ago

The issue here for me was that the rank 0 GPU was not returning from the forward pass, and for whatever reason the broadcast wasn't blocking. So the returned tmpdir was a bunch of garbage characters some of which couldn't be decoded.

Hello, I would ask that if you solve this problem. I also met the problem. It blocked the train lots of times and confused me.