appnexus / pyrobuf

A Cython alternative to Google's Python Protobuf library
Other
554 stars 76 forks source link

Disabled token 'MAP_FIELD' #151

Closed mattangus closed 5 months ago

mattangus commented 4 years ago

I am trying to compile this proto file:

syntax="proto3";

message Test{
    map<string, float> foo = 1;
}

I got this error:

$ python3 -m pyrobuf test.proto
generating test.proto
Exception while parsing test.proto
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/__main__.py", line 10, in <module>
    main()
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/__main__.py", line 6, in main
    compiler.compile()
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/compile.py", line 88, in compile
    self._compile_spec()
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/compile.py", line 121, in _compile_spec
    self._generate(source)
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/compile.py", line 133, in _generate
    msg_def = self.parser.parse_from_filename(filename, self.includes)
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/parse_proto.py", line 264, in parse_from_filename
    raise e
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/parse_proto.py", line 261, in parse_from_filename
    return cls(s).parse(fname=fname, includes=includes, disabled_tokens=disabled_tokens)
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/parse_proto.py", line 225, in parse
    imported['enums']
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/parse_proto.py", line 326, in _parse_message
    for token in tokens:
  File "/usr/local/lib/python3.5/dist-packages/pyrobuf-0.9.3-py3.5-linux-x86_64.egg/pyrobuf/parse_proto.py", line 157, in tokenize
    token_type, line + 1, self.lines[line])
AssertionError: Disabled token 'MAP_FIELD' found on line 4:     map<string, float> foo = 1;

protoc works on this example:

$ protoc test.proto --python_out=.
$ echo $?
0

I have installed from source instead of pip to make sure I had the latest features. Map support seems to be added by #122. Do I need to do anything specific to get pyrobuf to work with maps?

laike9m commented 3 years ago

Same question here

eric-saintetienne commented 3 years ago

Same issue with map<string, int64> and map<string, string>

BenTheKlein commented 3 years ago

Same question here

KimiNewt commented 2 years ago

Same with both :(