search

PaddlePaddle / models

Officially maintained, supported by PaddlePaddle, including CV, NLP, Speech, Rec, TS, big models and so on.
Apache License 2.0
6.9k stars 2.91k forks source link

ctr 下的dcn模型在云服务器运行失败 #3971

Open LBSIX opened 4 years ago

LBSIX commented 4 years ago

自己在本地跑dcn模型https://github.com/PaddlePaddle/models/tree/develop/PaddleRec/ctr/dcn可以成功运行,但是在云端服务器运行时，跑不通，第一次用的百度的aistudio，第二次租了其他的服务器，都是哦linux 环境。这是第二次的输出日志。还请百度的小哥哥帮我看看。本地跑的话太慢了，完全无法调试模型。前面有我发的第一次在aistudio跑的模型日志，下面是第二次的 epoch1 start ... Traceback (most recent call last): Traceback (most recent call last): Traceback (most recent call last): File "reader.py", line 96, in File "reader.py", line 96, in File "reader.py", line 96, in Traceback (most recent call last): File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe sys.stdout.write(self._gen_str(sample)) sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe BrokenPipeError: [Errno 32] Broken pipe sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "reader.py", line 96, in Traceback (most recent call last): File "reader.py", line 96, in Traceback (most recent call last): Traceback (most recent call last): File "reader.py", line 96, in File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin sys.stdout.write(self._gen_str(sample)) sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe BrokenPipeError: [Errno 32] Broken pipe sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "reader.py", line 96, in Traceback (most recent call last): File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin sys.stdout.write(self._gen_str(sample)) sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe (dl) root@562de4acacdf:/input/data/data/dcn# ;j;o -bash: 未预期的符号 `;' 附近有语法错误 [1]+ 已杀死 python -u local_train.py > train.log (dl) root@562de4acacdf:/input/data/data/dcn# python -u local_train.py > train.log & [1] 7719 (dl) root@562de4acacdf:/input/data/data/dcn# epoch1 start ... Traceback (most recent call last): Traceback (most recent call last): File "reader.py", line 96, in Traceback (most recent call last): Traceback (most recent call last): File "reader.py", line 96, in File "reader.py", line 96, in File "reader.py", line 96, in Traceback (most recent call last): Traceback (most recent call last): File "reader.py", line 96, in File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin criteo_dataset.run_from_stdin() criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin Traceback (most recent call last): criteo_dataset.run_from_stdin() File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin criteo_dataset.run_from_stdin() sys.stdout.write(self._gen_str(sample)) sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe BrokenPipeError: [Errno 32] Broken pipe sys.stdout.write(self._gen_str(sample)) sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin BrokenPipeError: [Errno 32] Broken pipe File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "reader.py", line 96, in Traceback (most recent call last): File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin Traceback (most recent call last): File "reader.py", line 96, in sys.stdout.write(self._gen_str(sample)) criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin BrokenPipeError: [Errno 32] Broken pipe criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "reader.py", line 96, in criteo_dataset.run_from_stdin() File "/usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/paddle/fluid/incubate/data_generator/init.py", line 135, in run_from_stdin sys.stdout.write(self._gen_str(sample)) BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> BrokenPipeError: [Errno 32] Broken pipe

bjjwwang commented 4 years ago

@wilhelmzh 求解答 谢谢

yolearn commented 3 years ago

有人找到问题吗