[Python] TypeError: A Message class can only inherit from Message

[Nukesor]

Hi!

I'm currently trying to run several Flask server instances with tensorflow inside of them. The server instances are managed by an uWSGI emperor. As long as I start the instance without the emperor, everything works perfectly fine:

uwsgi -s /tmp/solar.sock --manage-script-name --mount /=server:app --chown www-data --plugins python35 --venv ./venv

But as soon as i try to launch it utilizing the emperor, I get this error during app initialization:

Traceback (most recent call last):
  File "./server/__init__.py", line 6, in <module>
    from server.predict_service import PredictService
  File "./server/predict_service.py", line 1, in <module>
    import tensorflow as tf
  File "/opt/apps/solar/venv/lib/python3.5/site-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import *
  File "/opt/apps/solar/venv/lib/python3.5/site-packages/tensorflow/python/__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/opt/apps/solar/venv/lib/python3.5/site-packages/tensorflow/core/framework/graph_pb2.py", line 10, in <module>
    from google.protobuf import descriptor_pb2
  File "/opt/apps/solar/venv/lib/python3.5/site-packages/google/protobuf/descriptor_pb2.py", line 1716, in <module>
    __module__ = 'google.protobuf.descriptor_pb2'
TypeError: A Message class can only inherit from Message

I really don't know what causes this to happen. I know that this is a rare setup, but I would really appreciate every help I could get on this topic.

[Cristian-Malinescu]

I'm not so sure it is protobuf problem. It happens to me in the Spyder's iPython console also when trying to run the code for more than once, the 1st run executes with no errors.

`... tensorflow.python.platform.sysconfig, tensorflow.python.platform.test, tensorflow.python.client.device_lib, tensorflow.core.framework.device_attributes_pb2, tensorflow.python.framework.test_util, tensorflow.python.platform.googletest, tensorflow.python.platform.benchmark, tensorflow.core.util.test_log_pb2, tensorflow.python.client.timeline, tensorflow.python.util.protobuf, tensorflow.python.util.protobuf.compare, tensorflow.python.ops.gradient_checker, unittest.mock, tensorflow.python.util.lazy_loader Traceback (most recent call last):

File "", line 1, in runfile('/opt/[...]/main.py', wdir='/opt/[...]')

File "/opt/[...]/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 688, in runfile execfile(filename, namespace)

File "/opt/[...]/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile exec(compile(f.read(), filename, 'exec'), namespace)

File "/opt/[...]/main.py", line 5, in import tensorflow as tf

File "/opt/[...]/lib64/python3.6/site-packages/tensorflow/init.py", line 24, in from tensorflow.python import *

File "/opt/[...]/lib64/python3.6/site-packages/tensorflow/python/init.py", line 52, in from tensorflow.core.framework.graph_pb2 import *

File "/opt/[...]/lib64/python3.6/site-packages/tensorflow/core/framework/graph_pb2.py", line 10, in from google.protobuf import descriptor_pb2

File "/opt/[...]/lib64/python3.6/site-packages/google/protobuf/descriptor_pb2.py", line 1771, in module = 'google.protobuf.descriptor_pb2'

TypeError: A Message class can only inherit from Message`

Same code works perfect when launched from command line python. I suspect the issue is due to some code cashing done either by Spyder/iPython or emperor.

[yananzhaotpg]

I have the same problem.

[Venkat-Rajgopal]

Same issue with me. 1st run in Spyder executes and later it crashes. I have to restart the kernel every time. Works fine from the command line. Any fix to this ?

[alanleemachinelearning]

I got the same problem, any way can solve it ?

[pbrumblay]

I received this error with the protobuf library when running within a python:3 container which appears to be using debian jessie (see: https://hub.docker.com/_/python/). I did not get the error after switching to python:3-alpine.

[majinyu666]

Update on 2017-10-15 I was working with default python3 on ubuntu. Now I got a virtualenv to work in, installed all the packages required without all the packages not required. Now it works just fine.

---

Original comment My code is import tensorflow as tf , yes, just one line. Only the first time will be OK, the next time running this will raise TypeError, unless i restart console.

Traceback (most recent call last):

File "", line 1, in runfile('/home/miles/Python/tensorflow/examples/tensorflow_core_tutorial.py', wdir='/home/miles/Python/tensorflow/examples')

File "/home/miles/.local/lib/python3.5/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile execfile(filename, namespace)

File "/home/miles/.local/lib/python3.5/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile exec(compile(f.read(), filename, 'exec'), namespace)

File "/home/miles/Python/tensorflow/examples/tensorflow_core_tutorial.py", line 2, in import tensorflow as tf

File "/home/miles/.local/lib/python3.5/site-packages/tensorflow/init.py", line 24, in from tensorflow.python import *

File "/home/miles/.local/lib/python3.5/site-packages/tensorflow/python/init.py", line 52, in from tensorflow.core.framework.graph_pb2 import *

File "/home/miles/.local/lib/python3.5/site-packages/tensorflow/core/framework/graph_pb2.py", line 10, in from google.protobuf import descriptor_pb2

File "/home/miles/.local/lib/python3.5/site-packages/google/protobuf/descriptor_pb2.py", line 1771, in module = 'google.protobuf.descriptor_pb2'

TypeError: A Message class can only inherit from Message

But directly import tensorflow in the console will always be OK, like this: in [1]: import tensorflow as tf in [2]: import tensorflow as tf in [3]: import tensorflow as tf in [4]: import tensorflow as tf ... So strange.. Using Spyder 3.2.3, Python 3.5.2, iPython 6.2.1, on Ubuntu-16.04-TLS

[bkanuka]

I'm having the same error, but on Python 2.7 on Ubuntu 16.04. Nothing to do with Spyder or tf, but it happens only at import like everyone else:

File "/usr/local/lib/python2.7/dist-packages/gmusicapi/__init__.py", line 4, in <module>
  from gmusicapi.clients import Webclient, Musicmanager, Mobileclient
File "/usr/local/lib/python2.7/dist-packages/gmusicapi/clients/__init__.py", line 5, in <module>
  from gmusicapi.clients.musicmanager import Musicmanager, OAUTH_FILEPATH
File "/usr/local/lib/python2.7/dist-packages/gmusicapi/clients/musicmanager.py", line 27, in <module>
  from gmusicapi.protocol import musicmanager, upload_pb2, locker_pb2
File "/usr/local/lib/python2.7/dist-packages/gmusicapi/protocol/musicmanager.py", line 24, in <module>
  from gmusicapi.protocol import upload_pb2, locker_pb2, download_pb2
File "/usr/local/lib/python2.7/dist-packages/gmusicapi/protocol/upload_pb2.py", line 10, in <module>
  from google.protobuf import descriptor_pb2
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/descriptor_pb2.py", line 1772, in <module>
  __module__ = 'google.protobuf.descriptor_pb2'
TypeError: A Message class can only inherit from Message

[AdrienDeliege]

I had the same problem with Spyder, which happened when it was trying to reload modules that were already loaded. I solved it by disabling the UMR (User Module Reloader) option in "preferences -> python interpreter" .

[Venkat-Rajgopal]

@AdrienDeliege That seems to be working for me.

[abramclark]

I'm having the same issue, but here's the bizarre part: it only happens in my wsgi app when requesting via SSL. I don't know how this could possibly make a difference to Python. I can't reproduce it on any other machine or from any other process except from an SSL request to apache2. My stack trace starts from my code at "from google.cloud.storage.client import Client" and ends with:

File "/usr/local/lib/python2.7/dist-packages/google/protobuf/duration_pb2.py", line 10,
in <module>
  from google.protobuf import descriptor_pb2
 File "/usr/local/lib/python2.7/dist-packages/google/protobuf/descriptor_pb2.py", line 1824, in <module>
  __module__ = 'google.protobuf.descriptor_pb2'
TypeError: A Message class can only inherit from Message

[chpohl]

@AdrienDeliege Also works for me, but I'd like to still use UMR. My current workaround is to add tensorflow to the UMR excluded modules under "preferences -> Console -> Advanced settings".

[diegolorenzetti]

For me what solved the problem was setting the line below on my Apache virtual host:

WSGIApplicationGroup %{GLOBAL}

[jovany-wang]

Same issue with me.

[lmapii]

I believe that I have encountered the same issue with my test framework, so I'm simply adding some information here.

The problem seems to be some other dependency to the protobuf module. For me it was google.cloud modules which don't seem to like to be re-imported. This is reproducible using the following snipped that uses a rollback-importer similar to the one used in pyunit (http://pyunit.sourceforge.net/notes/reloading.html):

import sys

class RollbackImporter:
    def __init__(self):
        self.previous_modules = sys.modules.copy()

    def rollback_imports(self):
        key_list = []
        for modname in sys.modules:
            if not modname in self.previous_modules.keys():
                key_list.append(modname)

        for modname in key_list:
            del(sys.modules[modname])

# rollback importer singleton
_ROLLBACK_IMPORTER = RollbackImporter()

def execute():
    global _ROLLBACK_IMPORTER
    import google.cloud.logging as logging_

    _ROLLBACK_IMPORTER.rollback_imports()
    _ROLLBACK_IMPORTER = RollbackImporter()
    import google.cloud.logging as logging_

if __name__ == "__main__":
    execute()

I've tested this using Python 3.7.1. If only google.protobuf is imported in the above snipped (instead of google.cloud.logging) the problem does not occur. I'll go forward on the Cloud SDK issue platform with this one but maybe it gives some insight here.

In my framework I now simply exclude all google.* modules from being re-imported.

[hanzigs]

After installing keras or tensorflow, close the spyder then reopen, the "TypeError: A Message class can only inherit from Message" issue will be gone, also try to disable the tools -> preferences -> python interpreter -> Enable UMR if required

[muhammed100119]

i have the same issue. What is the solution

File "C:\Users\100119\AppData\Roaming\Python\Python36\site-packages\tensorflow\core\framework\resource_handle_pb2.py", line 92, in module = 'tensorflow.core.framework.resource_handle_pb2'

TypeError: A Message class can only inherit from Message

[Shilpibhargava]

Same issue with me on relaunch of spyder the import works fine. I am importing google OR tools. from ortools.constraint_solver import routing_enums_pb2 from ortools.constraint_solver import pywrapcp

[muhammed100119]

Problem: Same issue with me on relaunch of spyder the import works fine. I am importing google OR tools. from ortools.constraint_solver import routing_enums_pb2 from ortools.constraint_solver import pywrapcp

solution Restart the kernel

[tst-mbiancalana]

Just wanted to add some more info here. My version of protobuf==3.0.0.

I had this same error come up in some unit tests, and isolated it for my case. I have a dependency on a third-party module that is outside of my repo and so not available when the unit tests are run in-place. To work around this, I mocked the whole third-party module by patching sys.modules (see example below). The end result, though, is that everything that is imported while that mock is in effect (between patch.start() and patch.end() does not remain in sys.modules afterwards. One of these things that do not remain is my protobuf module. The next time the same python interpreter tries to import that protobuf module, the TypeError discussed here is raised.

Simplified example code (put any generated protobuf module in place of mything_v2_pb2):

import sys
import mock

third_party_module_mock = mock.MagicMock()
patcher = mock.patch.dict(sys.modules, {"third-party": third_party_module_mock})
patcher.start()
import mything_v2_pb2
patcher.stop()

print("mything_v2_pb2 in modules: {}".format('mything_v2_pb2' in sys.modules))
import mything_v2_pb2

I suspect that this has to do with double-loading the compiled google.protobuf code (one of the Python extensions written in C), which is what the mock.patch is tricking Python into doing. I don't think these are unloaded by Python. See this thread in the Python mailing list for more details.

I'm not sure that it will, but I hope this helps in the other cases as well.

[karlkaebnick]

Is there any more information on this? (I am getting this now working in the following environment: protobuf==3.11.2 python3.7.4 apache2.4

Update: I was able to make this go away by adding 'WSGIApplicationGroup %{GLOBAL}' to my apache config (as diegolorenzetti suggested, above)

[prakashscoopwhoop]

I had the same problem with uwsgi + Nginx + python2.7

File "/usr/lib/python2.7/site-packages/google/cloud/storage/init.py", line 38, in from google.cloud.storage.batch import Batch File "/usr/lib/python2.7/site-packages/google/cloud/storage/batch.py", line 29, in from google.cloud import _helpers File "/usr/lib/python2.7/site-packages/google/cloud/_helpers.py", line 33, in from google.protobuf import duration_pb2 File "/usr/lib64/python2.7/site-packages/google/protobuf/duration_pb2.py", line 69, in 'module' : 'google.protobuf.duration_pb2' TypeError: A Message class can only inherit from Message unable to load app 1 (mountpoint='scoopwhoop') (callable not found or import error) Fatal Python error: Py_EndInterpreter: not the last thread DAMN ! worker 10 (pid: 15115) died, killed by signal 6 :( trying respawn ... Respawned uWSGI worker 10 (new pid: 15146)

[Nukesor]

I'm just going to close this, as I'm personally no longer working with protobuf and since there doesn't seem to be a lot of official response :)

The ticket should still be helpful for other people as there seem to be a few workarounds for this problem.

In case you're still stumbling over this ticket, please open a new one so you can provide a fresh view and active support instead of responding to a ~5 year old issue :D.