ProHiryu / albert-chinese-ner

使用预训练语言模型ALBERT做中文NER
MIT License
467 stars 124 forks source link

更换自己数据后报错。 #40

Open Ricardo-ming opened 3 years ago

Ricardo-ming commented 3 years ago

新数据已经改为示例数据格式,labels名称和数量更换了,文件txt也是utf-8格式,但是依然会报错,百思不得其解: ERROR:tensorflow:Error recorded from evaluation_loop: 'utf-8' codec can't decode bytes in position 2844-2845: invalid continuation byte E1008 14:17:13.590271 139627806422848 error_handling.py:75] Error recorded from evaluation_loop: 'utf-8' codec can't decode bytes in position 2844-2845: invalid continuation byte INFO:tensorflow:evaluation_loop marked as finished I1008 14:17:13.590479 139627806422848 error_handling.py:101] evaluation_loop marked as finished WARNING:tensorflow:Reraising captured error W1008 14:17:13.590565 139627806422848 error_handling.py:135] Reraising captured error Traceback (most recent call last): File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call return fn(*args) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn target_list, run_metadata) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence [[{{node IteratorGetNext}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "albert_ner.py", line 848, in File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/platform/app.py", line 40, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/absl/app.py", line 312, in run _run_main(main, args) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/absl/app.py", line 258, in _run_main sys.exit(main(argv)) File "albert_ner.py", line 788, in main File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3056, in evaluate rendezvous.raise_errors() File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/error_handling.py", line 136, in raise_errors six.reraise(typ, value, traceback) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/six.py", line 719, in reraise raise value File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3051, in evaluate name=name) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 480, in evaluate name=name) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 522, in _actual_eval return _evaluate() File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 511, in _evaluate output_dir=self.eval_dir(name)) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1619, in _evaluate_run config=self._session_config) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/training/evaluation.py", line 272, in _evaluate_once session.run(eval_ops, feed_dict) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/training/monitored_session.py", line 754, in run run_metadata=run_metadata) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/training/monitored_session.py", line 1259, in run run_metadata=run_metadata) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/training/monitored_session.py", line 1360, in run raise six.reraise(original_exc_info) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/six.py", line 719, in reraise raise value File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/training/monitored_session.py", line 1345, in run return self._sess.run(args, *kwargs) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/training/monitored_session.py", line 1418, in run run_metadata=run_metadata) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/training/monitored_session.py", line 1176, in run return self._sess.run(args, **kwargs) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 956, in run run_metadata_ptr) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1180, in _run feed_dict_tensor, options, run_metadata) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run run_metadata) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1378, in _do_call message = error_interpolation.interpolate(message, self._graph) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/framework/error_interpolation.py", line 454, in interpolate common_prefix = traceback_files_common_prefix(tagged_ops) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/framework/error_interpolation.py", line 358, in traceback_files_common_prefix for frame in op.traceback: File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 2309, in traceback return tf_stack.convert_stack(self._traceback) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/util/tf_stack.py", line 199, in convert_stack return tuple(_tuple_generator()) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/site-packages/tensorflow_core/python/util/tf_stack.py", line 189, in _tuple_generator line = linecache.getline(filename, lineno, frame.globals) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/linecache.py", line 16, in getline lines = getlines(filename, module_globals) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/linecache.py", line 47, in getlines return updatecache(filename, module_globals) File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/linecache.py", line 137, in updatecache lines = fp.readlines() File "/home/mingzhen/anaconda3/envs/albert/lib/python3.6/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 2844-2845: invalid continuation byte

fany198 commented 1 year ago

一直报以下错误,请问大佬知道是怎么回事,要如何解决这个问题吗? tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for albert_base_zh/albert_model.ckpt

ProHiryu commented 3 months ago

没有检查点,先去下bert的参数