报错FileNotFoundError: [Errno 2] No such file or directory: '/app/work_dirs/chatglm2_6b_qlora_lawyer_e3_copy/20240514_035914/vis_data/eval_outputs_iter_499.txt' #685
修改tensorboard的日志目录后,当微调step执行到save_step时,报错FileNotFoundError: [Errno 2] No such file or directory: '/app/work_dirs/chatglm2_6b_qlora_lawyer_e3_copy/20240514_035914/vis_data/eval_outputs_iter_499.txt'.
def _save_eval_output(self, runner, eval_outputs):
save_path = os.path.join(runner.log_dir, 'vis_data',
f'eval_outputs_iter_{runner.iter}.txt')
with open(save_path, 'w', encoding='utf-8') as f:
for i, output in enumerate(eval_outputs):
f.write(f'Eval output {i + 1}:\n{output}\n\n')
修改tensorboard的日志目录后,当微调step执行到save_step时,报错FileNotFoundError: [Errno 2] No such file or directory: '/app/work_dirs/chatglm2_6b_qlora_lawyer_e3_copy/20240514_035914/vis_data/eval_outputs_iter_499.txt'.
debug发现是在到达save_step后,xtuner调用evaluate_chat_hook.py的_save_eval_output方法:
在该方法中,写死了vis_data目录,而在配置文件中改变了tensorboard的save_dir后,work_dir下的runner.log_dir不会产生vis_data目录,引起报错。
使用的复现配置文件如下: