search

devo-ps / pipelines

Build pipelines for automation, deployment, testing...
MIT License
119 stars 13 forks source link

Need fix logger stdout #107

Closed zbal closed 2 years ago

zbal commented 4 years ago

plugins/stdout_logger.py - L70 ; fails on writing the logs

Dirty approach - try / except around it.

Example of failure: 

Dec 10 10:28:09 builder pipelines: Traceback (most recent call last):
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/tornado/web.py", line 1512, in _execute
Dec 10 10:28:09 builder pipelines: result = yield result
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/tornado/gen.py", line 1055, in run
Dec 10 10:28:09 builder pipelines: value = future.result()
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/tornado/concurrent.py", line 238, in result
Dec 10 10:28:09 builder pipelines: raise_exc_info(self._exc_info)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/tornado/gen.py", line 1063, in run
Dec 10 10:28:09 builder pipelines: yielded = self.gen.throw(*exc_info)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/pipelines/api/utils.py", line 160, in _run_pipeline
Dec 10 10:28:09 builder pipelines: yield runner.run(user_context)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/tornado/gen.py", line 1055, in run
Dec 10 10:28:09 builder pipelines: value = future.result()
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/concurrent/futures/_base.py", line 398, in result
Dec 10 10:28:09 builder pipelines: return self.__get_result()
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/concurrent/futures/thread.py", line 55, in run
Dec 10 10:28:09 builder pipelines: result = self.fn(*self.args, **self.kwargs)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/pipelines/api/utils.py", line 64, in run
Dec 10 10:28:09 builder pipelines: return self.pipe.run()
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/pipelines/pipeline/pipeline.py", line 175, in run
Dec 10 10:28:09 builder pipelines: self.plugin_mgr.trigger('on_pipeline_start', pipeline_context)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/pipelines/plugin/manager.py", line 22, in trigger
Dec 10 10:28:09 builder pipelines: ret = cb(*args)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/pipelines/plugins/stdout_logger.py", line 81, in on_pipeline_start
Dec 10 10:28:09 builder pipelines: self.write('Pipeline started. vars: %s' % custom_var_serializer(pipeline_vars))
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/pipelines/plugins/stdout_logger.py", line 70, in custom_var_serializer
Dec 10 10:28:09 builder pipelines: log_str = yaml.safe_dump(v, default_style=False, default_flow_style=False)[:-1]
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump
Dec 10 10:28:09 builder pipelines: return dump_all([data], stream, Dumper=SafeDumper, **kwds)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all
Dec 10 10:28:09 builder pipelines: dumper.represent(data)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent
Dec 10 10:28:09 builder pipelines: node = self.represent_data(data)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data
Dec 10 10:28:09 builder pipelines: node = self.yaml_representers[data_types[0]](self, data)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict
Dec 10 10:28:09 builder pipelines: return self.represent_mapping(u'tag:yaml.org,2002:map', data)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping
Dec 10 10:28:09 builder pipelines: node_value = self.represent_data(item_value)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data
Dec 10 10:28:09 builder pipelines: node = self.yaml_representers[data_types[0]](self, data)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list
Dec 10 10:28:09 builder pipelines: return self.represent_sequence(u'tag:yaml.org,2002:seq', data)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence
Dec 10 10:28:09 builder pipelines: node_item = self.represent_data(item)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data
Dec 10 10:28:09 builder pipelines: node = self.yaml_representers[None](self, data)
Dec 10 10:28:09 builder pipelines: File "/opt/venv/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined
Dec 10 10:28:09 builder pipelines: raise RepresenterError("cannot represent an object", data)
Dec 10 10:28:09 builder pipelines: RepresenterError: ('cannot represent an object', DotMap(committer=DotMap(username=u'web-flow', name=u'GitHub', email=u'noreply@github.com'), added=[], author=DotMap(username=u'zbal', name=u'Vincent Viallet', email=u'vincent.viallet@gmail.com'), distinct=True, timestamp=u'2019-12-10T09:52:17+08:00', modified=[u'README.md'], url=u'https://github.com/Wiredcraft/xxxxx/commit/81efb979f591730bf4ffe9c9a42e41ce984b85fd', tree_id=u'01055ac98bb5224025c37d3d360977ad12b2053b', message=u'Update README.md', removed=[], id=u'81efb979f591730bf4ffe9c9a42e41ce984b85fd'))

/cc @JuhaS

zbal commented 4 years ago

related to https://github.com/yaml/pyyaml/issues/177 ?

kaleocheng commented 2 years ago

we updated the dotmap lib to the latest, it should be good now. closing as done