When exceptions occur, a helper grabs all the information and returns it. If during the process any non-serializable object is added, the whole process crashes as the local instance can't do anything about the object.
In the case below, this happened because the tmp directory was being captured, containing a binary monmap which caused a UnicodeError that had the undecodable contents of the monmap. The fix has to go through the values in remote/functions.py in the capture_exception helper, and ensure decoded values are set, as strings
File "/home/alfredo/.virtualenvs/ceph-medic/lib/python3.6/site-packages/remoto-1.1.2-py3.6.egg/remoto/backends/__init__.py", line 160, in wrapper
return self.channel.receive()
File "/home/alfredo/.virtualenvs/ceph-medic/lib/python3.6/site-packages/execnet-1.6.0-py3.6.egg/execnet/gateway_base.py", line 728, in receive
raise self._getremoteerror() or EOFError()
execnet.gateway_base.RemoteError: Traceback (most recent call last):
File "<string>", line 1374, in _save
KeyError: <class 'builtin_function_or_method'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1063, in executetask
File "<string>", line 1, in do_exec
File "<remote exec>", line 155, in <module>
File "<string>", line 708, in send
File "<string>", line 1345, in dumps_internal
File "<string>", line 1363, in save
File "<string>", line 1381, in _save
File "<string>", line 1462, in save_dict
File "<string>", line 1456, in _write_setitem
File "<string>", line 1381, in _save
File "<string>", line 1462, in save_dict
File "<string>", line 1456, in _write_setitem
File "<string>", line 1381, in _save
File "<string>", line 1462, in save_dict
File "<string>", line 1456, in _write_setitem
File "<string>", line 1379, in _save
DumpError: can't serialize <class 'builtin_function_or_method'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/alfredo/.virtualenvs/ceph-medic/bin/ceph-medic", line 7, in <module>
exec(compile(f.read(), __file__, 'exec'))
File "/home/alfredo/python/ceph-medic/bin/ceph-medic", line 6, in <module>
main.Medic()
File "/home/alfredo/python/ceph-medic/ceph_medic/main.py", line 45, in __init__
self.main(argv)
File "/home/alfredo/python/ceph-medic/ceph_medic/decorators.py", line 46, in newfunc
return f(*a, **kw)
File "/home/alfredo/python/ceph-medic/ceph_medic/main.py", line 145, in main
parser.dispatch()
File "/home/alfredo/.virtualenvs/ceph-medic/lib/python3.6/site-packages/tambo-0.4.0-py3.6.egg/tambo/dispatcher.py", line 21, in dispatch
result = instance.main()
File "/home/alfredo/python/ceph-medic/ceph_medic/check.py", line 65, in main
collector.collect()
File "/home/alfredo/python/ceph-medic/ceph_medic/collector.py", line 194, in collect
metadata[node_type][hostname] = get_node_metadata(conn, hostname, cluster_nodes)
File "/home/alfredo/python/ceph-medic/ceph_medic/collector.py", line 124, in get_node_metadata
node_metadata['paths'] = collect_paths(conn)
File "/home/alfredo/python/ceph-medic/ceph_medic/collector.py", line 88, in collect_paths
path_metadata[p] = get_path_metadata(conn, p, **kw)
File "/home/alfredo/python/ceph-medic/ceph_medic/collector.py", line 105, in get_path_metadata
files[i] = conn.remote_module.stat_path(i, None, None, kw.get('get_contents'))
File "/home/alfredo/.virtualenvs/ceph-medic/lib/python3.6/site-packages/remoto-1.1.2-py3.6.egg/remoto/backends/__init__.py", line 170, in wrapper
raise RuntimeError(exc_line)
RuntimeError: DumpError: can't serialize <class 'builtin_function_or_method'>
When exceptions occur, a helper grabs all the information and returns it. If during the process any non-serializable object is added, the whole process crashes as the local instance can't do anything about the object.
In the case below, this happened because the
tmp
directory was being captured, containing a binary monmap which caused a UnicodeError that had the undecodable contents of the monmap. The fix has to go through the values inremote/functions.py
in thecapture_exception
helper, and ensure decoded values are set, as strings