appnexus / pyrobuf

A Cython alternative to Google's Python Protobuf library
Other
555 stars 71 forks source link

support referencing nested type definitions in field definitions #56

Open daa opened 8 years ago

daa commented 8 years ago

Currently pyrobuf does not support defining field in message with type defined in another message.

Example proto:

message M {
    optional string s = 1;

    message N {
        required int32 value = 1;
    }
}

message M2 {
    optional M.N nested_reference = 1;
}

google protoc succeeds:

$ protoc nestedref.proto --python_out=.
$ echo $?
0

pyrobuf fails:

$ python -m pyrobuf --out-dir y x/nestedref.proto 
generating x/nestedref.proto
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/daa/pyrobuf/pyrobuf/__main__.py", line 90, in <module>
    main()
  File "/home/daa/pyrobuf/pyrobuf/__main__.py", line 18, in main
    gen_message(args.source, out=args.out_dir, build=args.build_dir, install=args.install)
  File "/home/daa/pyrobuf/pyrobuf/__main__.py", line 62, in gen_message
    generate(fname, out, parser, templ_pxd, templ_pyx)
  File "/home/daa/pyrobuf/pyrobuf/__main__.py", line 80, in generate
    msgdef = parser.parse_from_filename(fname)
  File "pyrobuf/parse_proto.py", line 195, in parse_from_filename
    return self.parse(s, fname=fname)
  File "pyrobuf/parse_proto.py", line 175, in parse
    rep['messages'].append(self._parse_message(s, token, tokens, enums, imported['enums']))
  File "pyrobuf/parse_proto.py", line 210, in _parse_message
    for token in tokens:
  File "pyrobuf/parse_proto.py", line 147, in tokenize
    raise Exception("Unexpected character '%s' in line %d: '%s'" % (s[pos], line, s[pos-10:pos+10]))
Exception: Unexpected character 'o' in line 11: ' M2 {
    optional M'
scantwell commented 8 years ago

Should be noted that this goes for enumeration type as well.