Make sure messages from server are string

[vallsv]

Hi,

Here is a MR to make sure data from server are readable as string.

In some cases, depending on the deploy you can have supervisor using different version of python.

This enforce to use str as metadata.

I have patched it in order to use it with python 3. If you still want to support Python 2 there is still probably things to do, like skipping this new _decode_bytes function.

Let me know what you think about it.

See you,

Without this patch, here is some problems i had:

ERROR 2021-10-04 16:32:40,266 multivisor.server.web: Exception on /api/data [GET]
Traceback (most recent call last):
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/flask/app.py", line 2070, in wsgi_app
    response = self.full_dispatch_request()
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/flask/app.py", line 1515, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/flask/app.py", line 1513, in full_dispatch_request
    rv = self.dispatch_request()
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/flask/app.py", line 1499, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/multivisor/server/util.py", line 54, in wrapper_login_required
    return func(*args, **kwargs)
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/multivisor/server/web.py", line 47, in data
    return jsonify(app.multivisor.safe_config)
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/flask/json/__init__.py", line 348, in jsonify
    f"{dumps(data, indent=indent, separators=separators)}\n",
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/flask/json/__init__.py", line 129, in dumps
    rv = _json.dumps(obj, **kwargs)
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/flask/json/__init__.py", line 56, in default
    return super().default(o)
  File "/users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

And

WARNING 2021-10-04 17:50:38,957 multivisor.lbm16quartz: Unexpected error KeyError('description')

Which is about:
> /users/blissadm/conda/miniconda/envs/multivisor/lib/python3.7/site-packages/multivisor/multivisor.py(333)__eq__()
-> p1.pop("description")

[tiagocoutinho]

Hi @vallsv, Thanks for the patch.

[vallsv]

I don't know. If you think it's a better place, ill take a look.

[laurent-claustre]

Hi Tiago and Valentin, thanks for the fix, here at ESRF we are a bit in trouble keeping running both python2 and python3 multivisor/supervisor couple. With this fix we can keep running both versions, then take more time to upgrade to python3 our tons of installation.

[vallsv]

parse_obj sounds like to do the exact same thing as my function. Ill try to use it and make a test, if one already exists.

[vallsv]

Sounds like it is not working using parse_objcause it is recursive

(Pdb) import traceback; traceback.print_stack()
  File "/users/valls/workspace/multivisor.git/multivisor/multivisor.py", line 63, in run
    self.refresh()
  File "/users/valls/workspace/multivisor.git/multivisor/multivisor.py", line 136, in refresh
    self.update_info(info)
  File "/users/valls/workspace/multivisor.git/multivisor/multivisor.py", line 117, in update_info
    info = parse_obj(info)
  File "/users/valls/workspace/multivisor.git/multivisor/util.py", line 63, in parse_obj
    return {parse_obj(k): parse_obj(v) for k, v in obj.items()}
  File "/users/valls/workspace/multivisor.git/multivisor/util.py", line 63, in <dictcomp>
    return {parse_obj(k): parse_obj(v) for k, v in obj.items()}
  File "/users/valls/workspace/multivisor.git/multivisor/util.py", line 63, in parse_obj
    return {parse_obj(k): parse_obj(v) for k, v in obj.items()}
  File "/users/valls/workspace/multivisor.git/multivisor/util.py", line 63, in <dictcomp>
    return {parse_obj(k): parse_obj(v) for k, v in obj.items()}
  File "/users/valls/workspace/multivisor.git/multivisor/util.py", line 60, in parse_obj
    elif isinstance(obj, six.text_type):
  File "/nobackup/lvalls1/valls/Software/miniconda3/envs/multivisor/lib/python3.7/bdb.py", line 88, in trace_dispatch
    return self.dispatch_line(frame)

And then Process is found as sub element and as Process was inherited from dict, it totally break the stored objects.

[vallsv]

Here is an updated version.

• Renamed function into util.patch_dict
• Added unittest testing binary message from supervisor

[tiagocoutinho]

Looks really good! Thanks a lot of the improvement