yhcc / BARTNER

215 stars 22 forks source link

 在中文bart上尝试出现错误 #9

Closed liushz closed 2 years ago

liushz commented 2 years ago

想在中文数据集上使用模型,将bart换成了huggingface的chinese-bart,但一直报如下错误: Cannot set field:src_tokens as input, exception happens at the 0 value. Traceback (most recent call last): File "/workspace/BARTNER/train.py", line 135, in <module> data_bundle, tokenizer, mapping2id = get_data() File "/usr/local/lib/python3.6/dist-packages/fastNLP/core/utils.py", line 357, in wrapper results = func(*args, **kwargs) File "/workspace/BARTNER/train.py", line 127, in get_data data_bundle = pipe.process_from_file(paths, demo=demo) File "/workspace/BARTNER/data/pipe.py", line 218, in process_from_file data_bundle = self.process(data_bundle) File "/workspace/BARTNER/data/pipe.py", line 192, in process data_bundle.set_input('tgt_tokens', 'src_tokens', 'src_seq_len', 'tgt_seq_len', 'first') File "/usr/local/lib/python3.6/dist-packages/fastNLP/io/data_bundle.py", line 142, in set_input dataset.set_input(field_name, flag=flag, use_1st_ins_infer_dim_type=use_1st_ins_infer_dim_type) File "/usr/local/lib/python3.6/dist-packages/fastNLP/core/dataset.py", line 787, in set_input raise e File "/usr/local/lib/python3.6/dist-packages/fastNLP/core/dataset.py", line 784, in set_input self.field_arrays[name].is_input = flag File "/usr/local/lib/python3.6/dist-packages/fastNLP/core/field.py", line 371, in is_input self._check_dtype_and_ndim(only_check_1st_ins_dim_type=self._use_1st_ins_infer_dim_type) File "/usr/local/lib/python3.6/dist-packages/fastNLP/core/field.py", line 423, in _check_dtype_and_ndim raise e File "/usr/local/lib/python3.6/dist-packages/fastNLP/core/field.py", line 406, in _check_dtype_and_ndim type_0, dim_0 = _get_ele_type_and_dim(cell_0) File "/usr/local/lib/python3.6/dist-packages/fastNLP/core/field.py", line 56, in _get_ele_type_and_dim res = [_get_ele_type_and_dim(cell_i, dim) for cell_i in cell] File "/usr/local/lib/python3.6/dist-packages/fastNLP/core/field.py", line 56, in <listcomp> res = [_get_ele_type_and_dim(cell_i, dim) for cell_i in cell] File "/usr/local/lib/python3.6/dist-packages/fastNLP/core/field.py", line 84, in _get_ele_type_and_dim raise SetInputOrTargetException(f"Cannot process type:{type(cell)}.") fastNLP.core.field.SetInputOrTargetException: Cannot process type:<class 'NoneType'>. 才学习NER,请问该怎么解决?谢谢